Sorry I'm new to Codalab. There are 10 submissions for each participant. Can I decide which submission file to be shown or only the last one?
Posted by: yzyalbert @ Aug. 15, 2023, 9:54 a.m.Dear Yzyalbert,
our leaderboards are configured so that the best score is shown instead of last score.
This means,
- if your new submission beats your previous one, it will be on the leaderboard.
- if your previous submission scores better than the new one, it will remain on the leaderboard.
Hope it helps.
Best,
Fred.
If I submitted multiple language pairs in one submission file, how does the score for each language pair show on the leaderboard?
Posted by: yzyalbert @ Aug. 16, 2023, 7:38 a.m.Dear Yzyalbert,
the leaderboard will display a score for each language pair, and there will be a leaderboard for each metric.
Best,
Fred.
I'm still confused about the situation of multiple language pairs.
The best score for different language pairs may be shown in different submission versions after 10 submissions.
Each language pair will show its best score on the leaderboard among 10 submissions and the macro-average of all language pairs will be the macro-average of best scores of each language pair, even if best scores of different language pairs are in different submission versions.
Is my understanding correct?
Dear Yzyalbert,
instead of having a phase per language pair like in previous years, we opted for one single phase for all.
Each LP is still assessed individually, as stated by the output of the scoring programme. The leaderboard will allow to rank participants for each LP individually.
Your submission should thus contain your best predictions for each LP you wish to participate in.
Best,
Fred.
Thanks for the explanation.
Posted by: yzyalbert @ Aug. 16, 2023, 9:44 a.m.