Dear Organizing Committee,
I hope this email finds you well. I want to seek clarification regarding the annotation standards for Track 1 datasets to ensure our team aligns with the intended evaluation criteria. Upon analyzing the training dataset (pseudo-labeled by a pre-trained model), we observed significant annotation ambiguities. For instance, certain aircraft instances were inconsistently labeled under at least four different categories, and similar discrepancies appear frequently across the training data. While we understand that pseudo-labels may inherit model biases, this raises questions about the human-annotated validation sets:
1. **Annotation Criteria for Validation Sets**:
Could you clarify the definitive labeling standards for objects such as **aircraft** and **oil storage tanks**? Are there specific criteria to resolve ambiguities?
2. **Land Cover Class Delineation:**
How should participants distinguish between **Rangeland** and **Agricultural land**? Is this based on the characteristics of the surrounding background information or other criteria?
We recognize that annotator subjectivity may influence these decisions, but understanding the official guidelines would greatly assist in refining our methodology. Any documentation or examples illustrating these standards would be deeply appreciated.
Thank you for your time and support in advancing remote sensing research through this competition. We look forward to your guidance.
Posted by: YaeMiko @ Feb. 22, 2025, 11:35 a.m.Hi
Thank you for reaching out with your questions regarding the annotation standards for Track 1 datasets. We appreciate your thorough analysis and your efforts to align with the intended evaluation criteria. Please find our clarifications below:
1. Annotation Criteria for Validation Sets
Response: Aircraft are not included in our manually annotated validation and test sets. Oil storage tanks may be annotated as buildings, depending on their structural characteristics and surrounding context.
2. Land Cover Class Delineation
Response: Agricultural land includes areas dedicated to cultivating annual crops, perennial woody crops, and non-native vegetation for grazing. This class is characterized by identifiable boundaries and artificial patterns discernible in satellite or aerial imagery.
Rangeland consists of both naturally occurring and human-maintained vegetation areas. However, certain managed grass areas—such as grazing fields and sports fields—are excluded, while grass in parks and golf courses is included.
Thank you again for your interest in the competition.
Junshi
Posted by: Junshi @ Feb. 27, 2025, 12:41 p.m.