Hello!
I opened a thread long ago titled "Trouble reproducing the baseline's results".
My original post was:
"Hello!
Ok, so I've tried reproducing the baseline's results multiple times both on google colab and on other servers.
On the servers, where validation in the train_detector function is possible, the validation mAP reaches the mid-thirties much like the baseline submission.
Yet, all the submissions fail miserably.
I stress, these are runs of the provided notebook for 10 iterations. Moreover, I've tried other runs making sure the config is absolutely identical (included only fixing a slight discrepancy in the data pipeline). Also tried more/less iterations and various other things: all seem to fail when submitting.
Frustrating, as there's many things I was hoping to try. I must be missing something obvious.
Any pointers to what may be the cause would be appreciated.
Thanks.
G."
I then quickly deleted my post after finding the issue (this is a competition after all).
I won't mention what it is, but it's very minor (and in the config).
Just finally deciding to inform the MAFAT team after seeing multiple other threads raise the issue.
Again, to be absolutely clear, as is, when rerunning the provided baseline notebooks and submitting, the model fails. I encourage the MAFAT team to try this.
With a minor amendment to the config, it works.
G.
Posted by: gfreund @ March 27, 2023, 5:13 p.m.Thanks a lot ! It would be really appreciate if you are willing to share the errors in the config.
Best regards,
Bizhe
Hi,
Please refer to "Configuration changes in the baseline model notebook for
results reproducibility" post in the forum.
MAFAT Challenge Team