Dear Organizers;
First of all thank you for organizing this challenge. I know it is very hard and challenging for you. One of the reasons that I put man-hours for your challenges is NTIRE's fairness and good reputation. However on this challenge I find a couple aspects that can be better.
1)
First of all In the main codalab page the challenge is decribed as
"The aim is to devise a network that reduces one or several aspects such as runtime, parameters, FLOPs, activations, and depth of IMDN (https://arxiv.org/pdf/2009.06943.pdf) while at least maintaining PSNR of 29.00dB on validation datasets.
Note that for the final ranking and challenge winners we are weighing more the teams/participants improving in more than one aspect (runtime, parameters, FLOPs, activations, depths) over the provided reference solution."
From my understanding, A method needs to at least satisfy 29dB to be considered for the challenge however after the release of the challenge I can see 28.95 was the threshold If this was declared before hand I believe many of us could get a network with a better results and I believe many of us discarded different variations just because It could not reach 29dB level
2)
Although there are many ways a network can be considered efficient, the final results are only based on the timings. This mislead the teams for example some of them mostly focused on parameters while other mostly focused on FLOPS If this was the case since the beginning It should have been declared. I come to the conclusion that you are only looking for improving the speed after the results mail stating that
"Note that the aim of this challenge is to improve networks efficiencies for image SR. Thus, during the development phase, the participants need to maintain the PSNR of 29.00 dB on the validation set while <<SPEEDING UP>> the inference of the network as much as possible. The ranking is based on the average runtime on the validation and test set."
Regards
Posted by: deepernewbie @ April 4, 2022, 6:39 a.m.Não funciona, não está claro oq fazer
Posted by: Nathany @ Jan. 12, 2023, 12:39 p.m.