Homophobia/Transphobia Detection in social media comments:-LT-EDI@RANLP-2023

Organized by DravidianLangTech - Current server time: Jan. 9, 2025, 9:46 p.m. UTC

First phase

First phase
Feb. 28, 2023, midnight UTC

End

Competition Ends
Aug. 31, 2023, 8:07 a.m. UTC

 Shared Task on Homophobia/Transphobia Detection in social media comments: at LT-EDI@RANLP 2023

Participants will be provided with sentences in comment, extracted from social. Given a comments, a system must predict whether or not it contains any form of homophobia/transphobia. The seed data for this task is the Homophobia/Transphobia Detection dataset [1], a collection of comments from social media. The comments are manually annotated to show  whether the text contains homophobia/transphobia.

The participants will be provided development, training and test dataset in English, Spanish, Hindi, Tamil, and Malayalam. To download the data and participate, go to codalab and click “Participate" tab. As far as we know, this is the first shared task on Homophobia/Transphobia Detection.

Task:

This is a comment / post level classification task. Given a Youtube comment, the systems submitted by the participants should classify it'. To download the data and participate, go to the Participate tab.

As far as we know, this is the first shared task on Homophobia/Transphobia detection. 

 

Paper  name format should be: TEAM_NAME@LT-EDI@RANLP-2023: Title of the paper. 

Example: NUIG_ULD@LT-EDI@RANLP-2023: Homophobia/Transphobia Detection for Equality, Diversity, and Inclusion

For electronic submission of papers to LT-EDI@RANLP workshop please use this link:

Following are some general guidelines to keep in mind while submitting the working notes.
- Basic sanity check for grammatical errors and reported results
- Papers should have sufficient information for reproducing the mentioned results- Papers should follow the appropriate style (We will use RANLP 2023 style: details below)
- Check the papers for text reuse / Plagiarism. This includes self-plagiarism as well. We would like to stress this point as ACL is quite strict about it. Any paper found to have plagiarized content should be rejected without further consideration.
- Please ensure the author names do not have any salutations like Dr., Prof., etc in the final version
 
All submissions should be in Double column RANLP 2023 format. Authors should use one of the RANLP 2023 Templates below:
 
 
Reference:
@article{chakravarthi2022can,
  title={How can we detect Homophobia and Transphobia? Experiments in a multilingual code-mixed setting for social media governance},
  author={Chakravarthi, Bharathi Raja and Hande, Adeep and Ponnusamy, Rahul and Kumaresan, Prasanna Kumar and Priyadharshini, Ruba},
  journal={International Journal of Information Management Data Insights},
  volume={2},
  number={2},
  pages={100119},
  year={2022},
  publisher={Elsevier}
}

Evaluation Criteria

We accept the test results only through the google form. The google form can be accessed from (link will be given later)
 
Submission should be a zip file with your team name containing tsv files for individual langauges -  'teamname_language.tsv' e.g. the zip file may contain teamname_tamil.tsv, teamname_malayalam.tsv etc. The submission will be evaluated with a weighted average F1-score. The results should be submitted on the google form Link: 

Classification system’s performance will be measured in terms of weighted averaged Precision, weighted averaged Recall, and weighted averaged F-Score across all the classes. Participants are encouraged to check their system with Sklearn classification report https://scikit-learn.org/stable/modules/generated/sklearn.metrics.classification_report.html

 

The results should be submitted on the google form Link:

Terms and Conditions

By downloading the data or by accessing it any manner, you agree not to redistribute the data except for non-commercial and academic-research purposes. The data must not be used for providing surveillance, analyses or research that isolates a group of individuals or any single individual for any unlawful or discriminatory purpose.

You should cite this papers if you are using our data.

@article{chakravarthi2022can,
  title={How can we detect Homophobia and Transphobia? Experiments in a multilingual code-mixed setting for social media governance},
  author={Chakravarthi, Bharathi Raja and Hande, Adeep and Ponnusamy, Rahul and Kumaresan, Prasanna Kumar and Priyadharshini, Ruba},
  journal={International Journal of Information Management Data Insights},
  volume={2},
  number={2},
  pages={100119},
  year={2022},
  publisher={Elsevier}
}

 

Important Dates for shared task:

Task announcement: Feb 20, 2023

Release of Training data: Feb 28, 2023

Release of Test data: May 10, 2023

Run submission deadline: June 1, 2023   Extended to June 7,2023

Results declared: June 10, 2023

Paper submission:1 July 2023    10 July 2023 

Peer review notification: 5 August 2023

Camera-ready paper due: 20 August 2023

Workshop Dates: 7-8 September 2023

 

  • Bharathi Raja Chakravarthi, Insight SFI Research Centre for Data Analytics, School of Computer Science, University of Galway, Ireland

  • Malliga Subramanian, Kongu Engineering College, Tamil Nadu, India

  • Rahul PonnusamyInsight SFI Research Centre for Data Analytics, Data Science Institute, University of Galway, Ireland

  • Paul Buitelaar Insight SFI Research Centre for Data Analytics, Data Science Institute, University of Galway, Ireland

  • Miguel Ángel García-Cumbreras, Universidad de Jaén, Spain 

  • Salud María Jiménez-Zafra, Universidad de Jaén, Spain 

  • José Antonio García-Díaz,  Universidad de Murcia, Spain

  • Rafael Valencia-García, Universidad de Murcia, Spain

Student Volunteer:

Email:  bharathiraja.akr@gmail.com

Results for last year can be found below for three languages:

Task A:

Task B:

First phase

Start: Feb. 28, 2023, midnight

Competition Ends

Aug. 31, 2023, 8:07 a.m.

You must be logged in to participate in competitions.

Sign In