Hi all,
Codalab needs python scripts for evaluation. In most previous NTIRE Super-Resolution challenges, the PSNR/SSIM evaluation script will compute PSNR/SSIM values over RGB channels. In most top CV/ML conference/journal papers, we calculate PSNR/SSIM in Y channel. To make our results from the challenge be more consistent with those in main papers, we adopt to evaluate in Y channel.
Our current Python scoring script can produce the same PSNR values as that used in main CV papers. So that, the PSNR results here can be used in future main CV papers directly. But, due to some restrictions (e.g., cv2 is not available in the current Codalab evaluation, time limit for running results), we just use ssim from skimage.metrics to provide SSIM values for fast running. The SSIM values are not same as those used in main CV papers. We will use PSNR for the main evaluation metric.
If you have a good SSIM Python implementation (e.g., the speed is similar to that of SSIM implementation from BasicSR, avoid using cv2) that can produce the same results as those by using Matlab one, please kindly contact us. Thanks.
Best,
Organizers