Competition

Semeval 2024 Task9 BRAINTEASER-A Novel Task Defying Common Sense

Organized by YifanJiang - Current server time: Nov. 21, 2024, 7:08 a.m. UTC

Previous

Evaluation
Jan. 10, 2024, midnight UTC

Current

Post-Evaluation
Jan. 31, 2024, 11 p.m. UTC

End

Competition Ends
Never

Practice

Start: Sept. 9, 2023, midnight

Description: Duration: Until approximately 10 Jan 2024 (official dates to be announced). Data:Use of the official evaluation script on trial data. High limit for submissions to allow extensive testing. Public leaderboard to enable participants to verify their format and approach. Purpose: Helps participants prepare by checking formatting and initial performance.

Evaluation

Start: Jan. 10, 2024, midnight

Description: Duration: Approximately 10 Jan to 31 Jan 2024. Data: Employment of the official evaluation script and official test data. Restriction on submissions, with emphasis typically on the final valid submission on CodaLab. Leaderboard visibility is hidden. Purpose: Official assessment phase, focusing on competition and ranking outcomes.

Post-Evaluation

Start: Jan. 31, 2024, 11 p.m.

Description: Duration: Begins around 31 Jan 2024. Data: Continues the use of the official evaluation script and test data. The submission limit is set high again (e.g., 999). The public leaderboard is public. Purpose: Enables scoring of contrastive runs for detailed analysis in system description papers. Supports future analysis and scoring for systems interested in the task beyond SemEval-2024. Encourages ongoing research and analysis post-competition.

Competition Ends

Never

You must be logged in to participate in competitions.

Sign In
# Username Score
1 lulu13gjdfnglgr 1.000
2 caoyongwang 1.000
3 Maxine 0.975