Hello, I have a question for submitting on test phase.
Could you tell me the format of submission file? the hierarchical structure of it.
for example, I submitted on validation phase like 'submit/Places/val/Completion/000000.png'
Do I have to use same structure of validation phase like 'submit/WikiArt/val/...'?
Thanks for your great challenge.
Posted by: mmyy2513 @ March 29, 2022, 2 a.m.Hello,
As we are expecting results for both validation and test, yes, you can do it as 'submit/<dataset>/val/...' and 'submit/<dataset>/test/...'.
Thank you for your answer. I got it.
I have one more question,
Is there any method to see just my test score without factsheet or other stuff i must submit?
Hello, I think I did not get your question. For the test scores, it is not possible yet to obtain an score before we receive all the participants submissions, download them, and compute them locally.
Posted by: afromero @ March 30, 2022, 11:48 a.m.Hi
Then, can't I check the score on the checkboard during the test stage? Like the validation stage. There are currently four participants on the checkboard.
Posted by: dhkim @ March 30, 2022, 12:27 p.m.Hello, no you can’t because we will run it over the full set of images. As I mentioned in the last mail last week, that checkboard does not reflect the test set because it corresponds to the validation ground-truth, so any test result will yield wrong metrics. We just did that for convenience to not modify the codalab code.
Posted by: afromero @ March 30, 2022, 12:55 p.m.