Will codalab select the submission with the highest score, or does it choose the most recent submission?
Also, is it possible for us to check our scores on the test phase? It says that we are able to receive feedback by clicking "Download output from scoring step," but that just returns a blank screen.
Anyways, now that the competition is reaching its end, I wanted to say thank you for hosting this competition!
It was a fun and challenging task, and I look forward to reading other people's solution papers.
Glad to hear you've enjoyed the competition! Like the development phase, the best-performing submission will remain on the leaderboard and be used for final ranking. We will privately contact participants to inform them of their detailed results so that they can include these details in their submission papers.
Greg Holste | CXR-LT Organizer
Posted by: gholste @ July 15, 2023, 9:51 p.m.In short, I cannot see my scoring results in the test phase, and I have to wait for the organization to inform me privately. Therefore, it might be challenging to change the strategy, etc., as it's hard to know what will work. By the way, I'm still enjoying this competition very much!
Posted by: v1olet @ July 16, 2023, 6:57 a.m.Exactly, that is by design -- it is not good machine learning practice to change aspects of one's model based on final test set performance. The purpose of the development phase was to use the feedback on held-out validation data to inform model development. The test phase is meant to evaluate the quality of your final model(s) that you have developed during the development phase.
Greg Holste | CXR-LT Organizer
Posted by: gholste @ July 16, 2023, 1:36 p.m.