Semeval 2024 Task9 BRAINTEASER-A Novel Task Defying Common Sense

Organized by YifanJiang - Current server time: April 18, 2024, 11:35 a.m. UTC


Jan. 10, 2024, midnight UTC


Jan. 31, 2024, 11 p.m. UTC


Competition Ends


Start: Sept. 9, 2023, midnight

Description: Duration: Until approximately 10 Jan 2024 (official dates to be announced). Data:Use of the official evaluation script on trial data. High limit for submissions to allow extensive testing. Public leaderboard to enable participants to verify their format and approach. Purpose: Helps participants prepare by checking formatting and initial performance.


Start: Jan. 10, 2024, midnight

Description: Duration: Approximately 10 Jan to 31 Jan 2024. Data: Employment of the official evaluation script and official test data. Restriction on submissions, with emphasis typically on the final valid submission on CodaLab. Leaderboard visibility is hidden. Purpose: Official assessment phase, focusing on competition and ranking outcomes.


Start: Jan. 31, 2024, 11 p.m.

Description: Duration: Begins around 31 Jan 2024. Data: Continues the use of the official evaluation script and test data. The submission limit is set high again (e.g., 999). The public leaderboard is public. Purpose: Enables scoring of contrastive runs for detailed analysis in system description papers. Supports future analysis and scoring for systems interested in the task beyond SemEval-2024. Encourages ongoing research and analysis post-competition.

Competition Ends


You must be logged in to participate in competitions.

Sign In
# Username Score
1 lulu13gjdfnglgr 1.000
2 Ipouyall 0.975
3 Maxine 0.975