There are two model, which one is better?
a: 500G MACC 0.1s/image
b: 400G MACC 0.2s/image
I want to know how to get the final score with MACC and runtime.
Thank you!
Dear richlaji,
In general terms, one should be optimizing both metrics - so getting less GMACs but slower runtime is not a desired outcome - and should generally be a rare outcome. For evaluation we use runtime to ensure and double-check that the GMAC computation is sound. In cases where GMAC and runtime diverge greatly from one another (i.e. GMACS are lower but runtime is order of magnitudes higher) we will inspect and investigate what might be the reason for this on the source code of the model, GMAC computation and runtime measurement, and ultimately decide on a case-by-case basis based on each specific case.
Best regards,
Edu.