MAFAT Satellite Vision Challenge Forum

Go back to competition Back to thread list Post in this thread

> Exception: Execution time limit exceeded!

Hi
Submitted a model based on baseline model. just changed hyperparameters. affects training only.
No logic No ensamble
Used the baseline model notebook to compose submission. Got a model of size 225M

This is the second time my submission fail

why is it and how can I precheck locally. On my i9 Rtx 3090 machine things looks fine. Surely not 2 sec per image

Thanks

Moshe

Posted by: moshebeutel @ March 13, 2023, 7:28 a.m.

Yes, same thing here. I am pretty frustrated. I keep getting this message.
I managed to have the baseline run but as soon as I make changes, it fails because of Exception: Execution time limit exceeded!
I tried to add logs and trace but nothing is available on codalab (all logs empty, no trace shows, is this normal? or am I doing something wrong?)
Locally, on my RTX 3090 Ti, the inference on the training dataset takes 650 ms per image.

Posted by: aiguy @ March 13, 2023, 4:32 p.m.

I tested the baseline submission and it runs at 621 ms per image on the training dataset on my machine.
So not very far from what I have with some modifications.
So there might be something else which fails.... but still no luck with the logs.

Posted by: aiguy @ March 13, 2023, 7:46 p.m.

OK. I was able to succeed by removing some computation.
I guess that I must be very close to the cutoff time...

Posted by: aiguy @ March 14, 2023, 4:40 a.m.

Hi,
Unfortunately, we cannot reveal the extended logs.
To evaluate whether your model meets the runtime limit of the predictions, we suggest comparing the runtime of your model to the baseline's runtime on your local device (loading, collecting statistics, updating the model and prediction). If the duration is close to or less than the baseline's runtime, then the assumption is that your model meets the runtime limit of the predictions.
Neta,
MAFAT Challenge Team

Posted by: MAFAT_Challenge @ March 15, 2023, 9:09 a.m.

Hi,
I recommend examining the modifications you made (regarding the baseline model) and whether they can cause a difference in the runtime.
Modifications such as dividing and predicting on tiny patches of a frame, increasing the range of precision of the variables that can cause to higher computational complexity, etc.

Neta,
MAFAT Challenge Team

Posted by: MAFAT_Challenge @ March 16, 2023, 10:03 a.m.
Post in this thread