Hello,
The test phase for the primary round of the competition has started and will conclude one week from now. Please see the points below for important information regarding the test phase.
- Each team will be allowed 5 submissions total. CodaLab only has functionality for per-participant limits on the number of submissions, so we will enforce this by looking at the aggregate submissions across all members from a team and removing all submissions except for the first 5 successful submissions made by that team. To reduce the workload for organizers, we ask that team members coordinate with each other to avoid crossing the 5 submission limit.
- The leaderboard is private. The scores for submissions will not be visible on CodaLab (at least for the next week), and the best submission out of the 5 allotted to each team will determine that team's rank in the test leaderboard.
- The existing Zenodo pages for the individual tracks have been updated to include the test networks. These are available at the following links.
Detection: https://zenodo.org/record/7213750
Target Label Prediction: https://zenodo.org/record/7213751
Trigger Synthesis: https://zenodo.org/record/7213752
Evasive Trojans: https://zenodo.org/record/7213753
We will also release new Zenodo records containing only the test set networks within the next day.
- The validation phase for all tracks except the Evasive Trojans track will remain open during the test phase for teams that wish to continue developing their methods before submitting to the test set. (Note: the starter kit for the Evasive Trojans track contains code for self-evaluation, which teams may use to further develop their method during the test phase.)
All the best,
Mantas (TDC co-organizer)