Hi organizers,
I noticed a note about the challenge phase, which states that we can't develop the algorithm further. So in the challenge phase, must the results we submit be consistent with the development phase? How to verify this?
Best,
Jerry Zhu.
Hi Jerry,
Thank you for your question. To clarify, we cannot prevent you from further developing your algorithms. However, during the final Challenge Phase, the leaderboard will be hidden and you will not receive any feedback on the test sets, which is different from the validation set used in the Development Phase evaluation. Additionally, you will not be able to download the evaluation score on your submission page. We only release training sets for subjects. Validation and test sets are used to evaluate your submission on CodaLab without releasing them.
Once the phase switches to the Challenge Phase, the system will automatically switch to using the test set as the ground truth. This means that there will be no feedback on the validation set during the final Challenge Phase. As a result, you will not receive feedback from either the test or validation sets. You will only have access to the released training set with test input poses.
If you wish to continue improving your methods, you can still use toy examples that contain entire views. However, please note that if you focus solely on improving your methods using these toy examples (2 subjects), your methods may become biased. It is up to you to decide whether or not to spend more time developing your methods.
Given these restrictions and the limited number of final submissions allowed within a week, we recommend that participants focus on finalizing and submitting their final results.
I hope this helps! Let me know if you have any further questions.
Posted by: youngkyoonjang @ July 12, 2023, 3:53 p.m.Hi organizers,
Another question. Is the final ranking based on masked psnr as an evaluation indicator? Does this mask refer to the border mask or the face mask in the toy example?
Best,
Jerry Zhu.
Hi Jerry,
Yes, the final ranking is based on Masked PSNR. The mask refers to a face mask, not a border mask.
Posted by: youngkyoonjang @ July 18, 2023, 11:25 a.m.Hi organizers,
I've tried three commits and it keeps showing running so I'm not sure if I'm committing correctly, can you please check the status of my commit?
Thanks,
Jerry Zhu.
Hi Jerry,
Please see:
https://codalab.lisn.upsaclay.fr/forums/13233/1841/
https://codalab.lisn.upsaclay.fr/forums/13233/1880/
Please let me know if you want me to delete your submission which is held as a ‘running’ status, so I can delete it.
Posted by: youngkyoonjang @ July 19, 2023, 5:28 p.m.Thanks. Please delete all the submission with running status for us.
Posted by: jerry_zhu @ July 19, 2023, 5:46 p.m.Hi Jerry,
I have deleted your submissions that were in the ‘running’ status. Just in case, try submitting using the Baseline-only CodaLab, as it has a longer deadline. And please read carefully about the previous forum thread: https://codalab.lisn.upsaclay.fr/forums/13233/1841/
You should be able to upload at least the dummy_submission.zip file.
Hi organizers,
Yesterday my teammate created the final submissions on her own account and succeed finally.
So please remove my submission from the leaderboard, which may still with running status.
And to make valid submission, she replace the readme file with the original one in dummy submission.
If it is need, we can send the readme file to your email.
Thanks.
Jerry Zhu.
Hi Jerry,
We have hidden all of your submissions for the final Challenge Phase.
Posted by: youngkyoonjang @ July 20, 2023, 7:32 a.m.