Competition

Mapillary Vistas Object Detection Task

We are pleased to contribute the Mapillary Vistas dataset to the Robust Vision Workshop @ ECCV 2022: 2D Bounding Box Detection Task. The goal of this task is to provide 2D bounding box detection results for a subset of 37 object classes on MVD. Provided detection results enable counting and localisation of individual, class-specific object instances like e.g. the number of cars or pedestrians in an image. Details on the submission format are provided next.

This CodaLab evaluation server provides a platform to measure performance on the validation and test set, respectively. A slightly modified variant of the COCO Panoptic API is used to compute the primary metric used for ranking.

The submission format is similar to the one described on the COCO dataset, however, due to the difference in naming convention of files, we adopted the following format:

[{
"image_id": str,
"category_id": int,
"bbox": x,y,width,height,
"score": float,
}]

Please note that the value for image_id is a string (and should be filled with the image filename without extension) as opposed to an integer used for the COCO. The category_id is a 1-based integer mapping to the respective class label positions in the config.json file, found in the dataset zip file available for download. For example, class Bird is the first entry in the config file and corresponding instances should receive label category_id: 1 (rather than 0). In addition, please note that the config file contains also stuff classes, such that values for category_id are not continuously assigned from 1 to 37. The bounding box coordinates are floats and referenced based on the top-left of an image (0-based). Please round the coordinates to the closest integers for avoiding excessive size of the JSON file. To check the correctness of your submission format, please submit results for the validation set through the corresponding phase of this benchmark server.

All detection results should be submitted as a zipped, single json file and can be submitted to this benchmark server. Additional information can be taken from the COCO upload and result formats for bounding box detection, respectively. The main performance metric used is Average Precision (AP) computed on box-level detections per object category, and is averaged over a range of overlaps 0.5:0.05:0.95 (inclusive start and end) with ground truth boxes. A maximum of 256 object detections are considered per image.