Thank you for your interest in our Parallel Domain & Valeo Motion Segmentation Challenge! We understand that some participants are curious about the evaluation formula. While we cannot reveal the exact formula, we’d like to provide some insights to help you better understand the scoring process.
Our evaluation metric is designed to encourage the effective use of synthetic data and minimize the reliance on real-world data from Woodscape. The score is a combination of the weighted Intersection over Union (wIoU) and a penalty term associated with the usage of Woodscape images. The starter kit includes code to calculate wIoU without any penalty. The wIoU is computed using weights of 0.02 for static classes and 0.98 for motion classes, emphasizing the importance of motion segmentation in our challenge. The penalty term is applied based on the number of Woodscape images used for training, with the intention of promoting solutions that rely more on synthetic data.
Please note that you can use upto 1/4th of the available Woodscape images without incurring any penalty. This allowance is meant to encourage fine-tuning and domain adaptation strategies.
We hope this information helps clarify the evaluation process, and we encourage you to focus on leveraging synthetic data to build innovative solutions for motion segmentation.
Good luck, and we look forward to seeing your submissions!
Posted by: saravanabalagi @ April 18, 2023, 5:06 p.m.