Dear participants,
Thank you very much for competing in MAFAT WiFi Challenge.
The private leaderboard is now available at the results tab, and as usual in data science competitions - the results are interesting!
Please note that due to technical difficulties on the Codalab platform (unrelated to the models or the users), some submissions failed at the private stage. Therefore, we created new users for those specific participants and resubmitted their models.
The new usernames are the original usernames with _ in the end.
We want to congratulate the leaders.
In the following few weeks we will conduct a thorough validation of the leaders winning-eligibility. Once done, the validated leaders will officially become winners and the competition winners will be officially announced.
In the next few days we will contact the leading contestants with further details and requests.
MAFAT would like to thank all of the competitors for a great effort and great results.
We hope you enjoyed this MAFAT Challenge and we hope to see you soon in our future challenges (register here in order to receive updates regarding upcoming MAFAT Challenge competitions).
MAFAT Challenge Team
randomness in results ?
hi mafat team, can you please verify if there's no error while checking private scores? It might sound stupid, as public and private lbs are bound to vary in data science competitions, but for me the validation scores and the private scores are very much different, inspite of having a validation that takes care of different devices from different times being in the validation set.
Also Is it possible to release the private dataset with labels to cross -verify our results ? I think that will be super useful. Since MAFAT team has reported bugs, there might be some bugs induced in scoring too.
P.S: Nothing to take away from the winners, congratulations to all the top finishers.
Posted by: devnikhilmishra @ June 29, 2022, 12:59 p.m.Hi,
We can assure you that the leaderboard scores are subject to an extensive validation process. Unfortunately, we do not intend to release the private dataset at this time.
MAFAT Challenge Team
Ouch this hurts.
This is huge shakeup - after lots of hard work, seems private data is different.
We got judgement rather than good news from holy land ...
Oh well, so be it.
MAFAT thanks and congrats to the winners.
I followed mainly track 2 - seems harelr did something right, harelr? any comments?
Posted by: skaak @ June 29, 2022, 2:20 p.m.Also MAFAT - any comments about solutions / evaluation method?
I realised towards the end e.g. that my model never predicts a '3' so I desperately tried to tweak it to do this with limited time ...
Perhaps MAFAT you did some tests and checked that models produced results for all 0, 1, 2 and 3 and penalised models if they did not? Any comments welcome ...
Posted by: skaak @ June 29, 2022, 2:25 p.m.Hi Mafat,
Why do I see 4 entries in my submissions table?
I have submitted only 2 times (at 25\6) but I see 4 (2 at 25\6 and 2 at 28\6).
I think additional entries is when MAFAT scored your model.
Posted by: skaak @ June 29, 2022, 2:49 p.m."I think additional entries is when MAFAT scored your model." - Yes, that's right.
MAFAT Challenge Team