Dear participants,
We have just released the dev data for task 2, on top of the already released train and dev data. If you download now the "Public data" in the "Files" section, you will find the following files:
- TA1C_dataset_detection_train.csv: the training data for task 1.
- TA1C_dataset_detection_dev.csv: the dev data for task 1, without the expected labels.
- TA1C_dataset_spoiling_train.csv: the training data for task 2.
- TA1C_dataset_spoiling_dev.csv: the dev data for task 1, without the expected responses.
Participants now can send their submissions using the train data for training and predicting over the development data. Please notice that the submission must be a ZIP file containing one or two CSV files:
- For Task 1, the file name must be "detection.csv" and contain the columns "Tweet ID" and "Tag Value"
- For Task 2, the file name must be "spoiling.csv" and contain the columns "Tweet ID" and "Response"
If you are participating in both tasks, please include both files at the same time in your submission.
During the development phase, you can make any number of submissions you want.
Regards,
TA1C organizing team
One important detail to point out is that currently the evaluation script for task 2 only computes BLEU score, as is the only metric available in the scoring environment. Once this phase is over, we will calculate the remaining automatic metrics (ROUGE and BLEUscore) for the submissions, and release the results.
Regards,
TA1C organizing team