Hi, dear all,
Thanks for participating in our Challenge. Here is the reply to the questions raised by participants:
1. Does the score obtained by submitting results on CodaLab represent Top-1 accuracy?
The score obtained on Codalab is Top 1 accuracy.
2. Does the order of each line of data in the Submission.csv file affect the results?
We modify the evaluation code on the server so that now you can submit your results without aligning the sample order, you just need to make sure that the clip name is correct: for instance, 0001_2 stands for the 2nd MG clip from video sample 0001. Please see line 126 in imigue_gendata.py file.
3. Validating set is missing?
The validating set was only available with preprocessed version (raw data was empty), we re-upload the whole set and now the validating set (raw data) is available. Actually validating set is a part of the training set (sample 0001-0011), you can directly divide them from the training set.
4. The testing set is leaked out?
Some participants found that preprocessed version of the testing set (participants ) was in there, actually, it was for the sample usage and the data inside was incorrect.
5. The starting kit has some bugs?
There were some bugs relating to the index sampling and joint reading, now they are fixed in the latest version.
6. Empty skeleton data in some clips?
There are some clips with empty skeleton data for which the OpenPose did not work, you can just leave them as zero and gives random predictions for those clips.
6. How will the final evaluation be conducted?
Participants need to provide the source code for our organizers to confirm the final score, the final ranking will base on the scores reproduced by our organizers.
Please feel free to ask more questions by posting on this thread.
Best regards,
Haoyu Chen
Hello! I've noticed that the evaluation system has become very slow in providing results. The situation occurred after I submitted an answer with a formatting error. Prior to that, the system usually provided feedback within a few minutes. What could be the reason for this? Could you please give me some advice? Thanks in advance.
Posted by: ChenxiCui @ May 4, 2023, 6:16 a.m.