Hi,
I made my first submission today but couldn't get any score. When I check my subsmission, all logs are empty.
I thought it was maybe because I didn't use the "prepare_upload.sh" script to make my archive.
So, I used this script to make a second submission. Still, there is no score and logs are empty.
Can one of the organizers check what went wrong?
Thanks!
Posted by: _mahcih_ @ June 14, 2022, 9:30 p.m.Hi _mahcih_ ,
Thank you for letting us know!
It seems your submission is stuck in the 'submitted' state. Ordinarily, this means that it is queued (has not yet started running, so no logs).
We have checked our workers and while they are up and running, it seems there is a connection issue to the Codalab platform. This is probably on our side because there was some maintenance of our servers going on yesterday morning. We are working on fixing it and will keep you updated. In the meantime, you can use the scripts we provide for local evaluation. Sorry for the inconvenience and thank you for your patience.
Best,
Steven
Update: Codalab evaluation still down...
Our workers are still suffering from communication issues and submissions are currently not evaluated. Please use the local evaluation scripts until these issues are resolved. We will keep you updated. Sorry for the inconvenience.
Posted by: adriaens @ June 16, 2022, 9:59 a.m.Update: This issue should now be resolved :-)
Good luck and let me know if you would run into any problems!
Posted by: adriaens @ June 17, 2022, 10:15 a.m.Dear team,
My submission is successfully terminated but it has no score.
Any hints?
Hi Tak,
I see nothing wrong with your submission. It finished and got a score (test loss of best checkpoint) for each of the 80 tasks (= instances in DAC terminology).
To see your competition score (average rank across the 80 tasks/instances) you must submit your solution to the dev phase leaderboard (you can do so by pressing the + next to your submission in the participate tab (submit / results section) and pressing the "submit to leaderboard button". Your 80 scores will then be compared to those obtained by the other submissions on the leaderboard (your score can be found in the leaderboard (in the results tab) in the Avg Rank column, where lower is better).
Note that we will use more / different tasks for final evaluation (+ some participants may not have submitted their solution yet), so final rank can differ.
I hope that helps? If not, let me know!
Best,
Steven
Thanks.
How about my sub z-inter.zip ? it is almost similar to z-inter2.zip but failed ?
I do at first sight not see anything in the logs explaining why it failed, so I would probably try resubmitting it. However, since your most recent z-inter.zip submission finished successfully, I guess you already did and this issue is resolved?
Posted by: adriaens @ July 4, 2022, 4:44 p.m.yeah, I just re-submitted (same zip file) and it finished successfully.
Posted by: tak @ July 4, 2022, 6:03 p.m.