A kind request to organizers to share the official metric implementation if possible.
It is unclear from evaluation description whether images, than marked as non-"hermetically tagged" are excluding predictions for small vehicles from the evaluation or not.
In other worst, if frame is not "hermetically tagged", if model predicts small vehicles they will be counted as FP if metric does not respect this flag.
Can you please clarify the logic of using" hermetic_small_vehicle" in the metric.
Thanks!
Hi.
If a frame is not "hermetically annotated", and the model predicts small vehicles, then the metric will not consider these predictions. Meaning it will not be counted as FP/TP/FN.
Neta,
MAFAT Challenge Team