Hi team,
Could you please share the inference and scoring code for the testing?
How do you calculate inference time? Does the time limit include the time when loading the model or not?
Is it possible to use multiprocess to speed up the inference?
Thanks
Posted by: rasyidridha @ Feb. 20, 2023, 11:41 p.m.Hi,
You can see an example of mAP metric implementation in this link: https://github.com/CAPTAIN-WHU/DOTA_devkit/blob/master/dota_evaluation_task1.py.
As a rule of thumb, we consider the inference time limit as 2 seconds per image (1280X1280 pixels) times the number of images, and proper additional time for model loading (up to 1 GB).
The instance g4dn.xlarge is used for running and scoring your inference code. Your model receives one image and its corresponding metadata at a time (this part is not parallelable, however you can use multiprocessing techniques in your predict function). Please ensure the ability of this instance to support your desired run mode.
Neta,
MAFAT Challenge Team