Shared Task on Multilingual Bias and Propaganda Annotation in Political Discourse

LT-EDI at LDK 2025

The Bias and Propaganda Annotation task aims to advance the understanding and annotation of biased and propagandistic content in English and Tamil texts, with a focus on political discourse, such as the US Gender Policy and India’s Three Language Policy. Participants are required to develop annotation systems that annotate text based on the presence of bias or propaganda. Participants are encouraged to utilize any existing or new technologies as part of their annotation process. 

The shared task aims to serve as a collaborative platform where participants propose diverse methods for annotating and analysing the dataset. The primary goal is to create a shared corpus for multilayered annotation, developing annotation guidelines that reflect the diverse and often conflicting discourses surrounding this sensitive topic. Additionally, it seeks to cultivate the next generation of Natural Language Processing (NLP) researchers by equipping them with hands-on experience in working with raw data sources. 

Task1: Bias and Propaganda Annotation in English 

• Sub Task 1: The goal is to focuses on annotating contents related to Trump's US gender policy against transgender individuals. The task is to annotate based on the bias and propaganda guidelines in English texts that discuss or analyse this policy. There are totally 6 bias labels and 4 propaganda labels.
• Sub Task 2: Annotate the content of YouTube comments related to the Ukraine-Russia war in English. The task involves categorizing the comments based on bias and propaganda, following established guidelines for analyzing bias and propaganda in English texts. There are totally  8 bias labels and 4 propaganda labels.

Task 2: Bias and Propaganda Annotation in Tamil 

• The goal of  task 2 is to provide annotating content related to the Three Language Policy/National Education Policy related issues. The task is to annotate based on the bias and propaganda guidelines in Tamil texts that discuss or analyse this policy. There are totally 7 bias labels and 4 propaganda labels.

Authors' Kit for Participation

• Account Setup:

• CodaLab Account: All participants must create an account on CodaLab to participate. This platform will be used for accessing competition details and datasets (click HERE to create an account).

• Registration Process:

• Joining the Competition: After logging into CodaLab, navigate to the ‘Participate’ tab and select ‘Register’ to join the competition.
• Access to Data: Once your registration is approved, you will gain access to the necessary datasets through CodaLab in the ‘Participate’ tab.Subtask 1 focuses on annotating content related to Trump's US gender policy concerning transgender individuals. The task involves annotating based on bias and propaganda guidelines in English texts that discuss or analyze this policy. There are a total of 6 bias labels and 4 propaganda labels.

• Result Submission:

• Procedure: Results must be submitted via a Google Form (it will be available in the ‘Evaluation’ or ‘Participate’ tab)  provided by the organizers. Detailed submission guidelines are available in the ‘Evaluation’ tab.
• Note: Do not submit your results directly on the CodaLab platform.

• Timeliness: It is crucial to follow the submission deadlines strictly. Late submissions may not be considered for evaluation.

Paper Submission Guidelines

• Format and Naming Convention:

• Title Format: The title of the paper should follow this format: TEAM_NAME@LT-EDI-2025: Title of the paper. For example, UOG@LT-EDI-2025: Bias and Propaganda Annotation
• Authors' Names: Ensure that the authors' names on the paper match those entered in the Google Form during result submission.

• Length and Content:

• Document Length: The paper should be a maximum of 4 pages, excluding appendix and citations (references).
• Paper Template: Paper submissions must use the LaTeX templates.
• Note: Poster and demo submissions should be no longer than 4 pages (plus an unlimited number of pages for references and ethics/broader impact statements).

• References and Citations:

• Citing the Dataset: Make sure to cite the dataset and any overview papers relevant to the competition. BibTeX entries for these citations will be provided below.

• Additional Points:

• Innovation and Originality: Submissions that demonstrate innovative approaches and original research will be highly valued.

  The following are some general guidelines to remember while submitting the working notes.

• Basic sanity check for grammatical errors and reported results
• Papers should have sufficient information for reproducing the mentioned results.- Papers should follow the appropriate style.
• Check the papers for text reuse / plagiarism. This includes self-plagiarism as well. We want to stress this point, as ACL is quite strict about it. Any paper found to have plagiarized content should be rejected without further consideration.
• Please ensure the author names do not have any salutations like Dr., Prof., etc, in the final version

 All submissions should be in Double-column LDK 2025 format. Authors should use the LDK 2025 templates below:

• LaTeX Template:  ACL Conference Template (Overleaf)

• Paper Submission:  Submit via OpenReview

📧 If you have any queries, contact us 📧  

 shunmugapriya.mc@gmail.com and bharathiraja.akr@gmail.com

 Reference: Cite the below reference

@inproceedings{chakravarthi-etal-2023-exploring,
    title = "Exploring Techniques to Detect and Mitigate Non-Inclusive Language Bias in Marketing Communications Using a Dictionary-Based Approach",
    author = "Chakravarthi, Bharathi Raja  and
      Kumaresan, Prasanna Kumar  and
      Ponnusamy, Rahul  and
      McCrae, John P.  and
      Comerford, Michaela  and
      Megaro, Jay  and
      Keles, Deniz  and
      Feremenga, Last",
    editor = "Mitkov, Ruslan  and
      Angelova, Galia",
    booktitle = "Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing",
    month = sep,
    year = "2023",
    address = "Varna, Bulgaria",
    publisher = "INCOMA Ltd., Shoumen, Bulgaria",
    url = "https://aclanthology.org/2023.ranlp-1.99/",
    pages = "918--925"}