Hello, am I only allowed to train with NYU Depth V2 in Track 2? Are simulated damaged images also prohibited? Will the official code reproduction be done to ensure fairness of the race?
Posted by: Suzally @ Feb. 22, 2023, 4:36 p.m.Hi Suzally,
Yes. The participants should follow the exact same instructions in preparing the training and evaluation data sets, as listed on this page: https://github.com/ldkong1205/RoboDepth/tree/main/competition.
In addition to the training data, there are some other useful techniques that you can use for this competition. You can refer to the discussion in the following issue as an example: https://github.com/ldkong1205/RoboDepth/issues/6.
Thanks!
Posted by: LingdongKong @ Feb. 22, 2023, 11:49 p.m.Is TTA(Test-Time Augmentation) allowed?
Posted by: Suzally @ March 22, 2023, 2:11 a.m.Thanks for your question!
Test-time augmentation is a common trick in academic competitions and is also allowed in this competition.
You can find some other clarifications on this issue: https://github.com/ldkong1205/RoboDepth/issues/6
Posted by: LingdongKong @ March 22, 2023, 1:05 p.m.Hello. I noticed that many state-of-the-art depth estimation models, such as GLPDepth, use the following data augmentations:
*********************************************************
*********************************************************
basic_transform =
[A.HorizontalFlip(),
A.RandomCrop(crop_size[0], crop_size[1]),
A.RandomBrightnessContrast(),
A.RandomGamma(),
A.HueSaturationValue()]
aug = A.Compose(transforms=self.basic_transform,
additional_targets=additional_targets)
augmented = aug(image=image, depth=depth)
*********************************************************
*********************************************************
May I ask if using the open-source .pth file of this model, which was trained on the NYU-Depth dataset, is compliant?
Hi Suzally, thanks for asking!
For your question:
- We limit the use of data augmentation because we would like to ensure that the comparisons are fair among all participants.
- For the configuration of data augmentation, we recommend all participants follow those have been defined in MonoDepth2, i.e.,
```
self.brightness = (0.8, 1.2)
self.contrast = (0.8, 1.2)
self.saturation = (0.8, 1.2)
self.hue = (-0.1, 0.1)
transforms.ColorJitter.get_params(
self.brightness, self.contrast, self.saturation, self.hue
)
```
- We will treat augmentations other than the above as "external" ones and we would expect participants will not use them in this competition.
Should you have any other questions, please let us know at your earliest convenience!
Regards,
The RoboDepth Organizing Team
I'm sorry I didn't notice this requirement. Is this requirement mandatory? Or is it a ranking priority. I don't remember seeing this requirement in the github toolkit.
Posted by: Suzally @ March 30, 2023, 4:36 p.m.Hi Suzally,
Sorry for the misunderstanding. The above requirements are for Track 1: self-supervised monocular depth estimation.
For Track 2, please follow the default settings in Monocular-Depth-Estimation-Toolbox.
The requirements related to data augmentation are mandatory. We would expect participants to follow the rules and we will check the reproducibility of potential winner solutions to ensure that the rules are followed. We have highlighted this in the Terms & Conditions section. Please have a look.
Regards,
The RoboDepth Organizing Team
At the https://github.com/ldkong1205/RoboDepth/issues/6 you mentioned
“Designing new data augmentation techniques to improve the robustness is interesting. We encourage participants to explore this direction. However, as mentioned in the terms & conditions of this competition, "any use of the 18 corruption types designed in this benchmark is strictly prohibited, including any atomic operation that is comprising any one of the mentioned corruptions". Please pay extra attention to this to avoid a possible penalty. Also, some recent works unveil that, although using a specific corruption operation as the augmentation during training might improve the performance of this corruption type, it is likely that the final scores will become lower since it tends to hurt the model's generalizability on other corruptions.”
That means we need to design our own data augmentation and can't use functions other than the above to transform the image properties directly?
Hi USTC-IAT-United,
Your understanding is correct. We encourage new designs but we would not want to see a violation of fairness. Therefore, you must be very careful in adding augmentations during the model training. Here are examples of CAN and CANNOT:
- CAN: Newly proposed data augmentation that is overtly different from conventional ones and does not make any "shortcuts" for simulating evaluation data distribution. Possible new augmentations might include image-masking, copy-pasting, etc., which are not the atomic operations that comprise any of the corruptions.
- CANNOT: Directly adjust conventional data augmentations to mimic similar cases of the evaluation data.
We hope all participants can understand our intention (a fair competition) and we would encourage every participant to follow the terms & conditions.
Regards,
The RoboDepth Organizing Team
The data augmentation of "https://github.com/zhyever/Monocular-Depth-Estimation-Toolbox" includes cropping images to a fixed size. However, I am limited by the model input and I can only use other resolutions for training. I would like your permission to operate this way.
Posted by: Suzally @ April 3, 2023, 12:29 p.m.Hi @Suzally,
Don't worry. Image cropping is allowed in this competition, as it does not constitute any of the 18 natural corruptions in the benchmark.
Regards,
The RoboDepth Organizing Team