Please check the starter kit (piq23_starter_kit) in Files / Submission / Participate
Given an input image, the model must produce a quality score JOD.
- ntire24_overall_scene_train.csv includes the filenames of the training images with the corresponding JOD
- ntire24_overall_scene_test.csv Includes the filenames of the public testing images, with some metadata that you can use.
You need to submit a csv (any other format will fail!) in the starter kit you can find pred_piq23.zip
The csv must be compressed into a zip file. The submission should have [1486 rows x 2 columns], the columns must be "Prediction" and "Class".
Each row in the csv corresponds to an image in the test file (see ntire24_overall_scene_test.csv).
We use mainly Spearman’s rank correlation to compare the predicted scores with the ground-truth scores (ranking).
We will include the Pearson correlation coefficient (PCC) and Kendall rank correlation coefficient in the final report.
Model's should have similar complexity as HyperIQA and MUSIQ. The model's should run on consumer GPUs with 8Gbs VRAM. See the requirements in the Evaluation section.
enjoy,
Posted by: nanashi @ Feb. 5, 2024, 12:02 a.m.Direct download here: https://codalab.lisn.upsaclay.fr/competitions/17311#participate-get_starting_kit
It includes a notebook to do your 1st sample submission
Posted by: nanashi @ Feb. 5, 2024, 12:44 a.m.Excuse me, could you take a look at the forums? We have a lot of problems that we can't submit the final file.
Posted by: Panmoon @ March 21, 2024, 3:12 a.m.