ASVspoof 5: Track 2

Spoofing-robust Automatic Speaker Verification

This is the page for submitting results to the Track 2, Open Condition, of the ASVspoof 5 a spoofing-robust automatic speaker verification (SASV) task. SASV systems must compare an unlabeled probe (test) utterance to an enrollment utterance(s) of known target speakers. SASV systems developed by participants will be benchmarked using a mix of three types of trials-target bonafide (target), non-target bonafide (non-target), and spoofing attack (spoof). SASV systems should accept targets only. In the Open Condition, the use of external data and pre-trained models is permitted, provided that their use is declared and described clearly in the system description accompanying any submission. Please, check the rules and the data usage of the open condition in the ASVspoof 5 evaluation plan.

You must be logged in to participate. The registration procedure is described in the ASVspoof 5 evaluation plan.

How to Submit?

1. Download development audio dataset.
2. Train, develop, and tune your spoofing counter-measure (CM), ASV, or SASV systems according to the protocols of ASVspoof 5.
3. Run your system on the evaluation data. Produce scores for each audio file (high: bonafide; low: spoof). Store it as a text file in tsv (tab-separated values) format.
4. Zip the score file without a parent directory structure, e.g., using a command equivalent to

     $ zip submission.zip score.tsv

5. Submit the zipped archive on this platform at: Participate → Submit / View Results → Submit.
   Please, provide method description details on the submission page.

Submission Score Format

Updated format: 4 July 2024

Considering possible scenarios for submitting CM, ASV, or End2End SASV system scores, we define two approaches and corresponding score formats. The score file has five-columns structure <spk> <filename> <cm-score> <asv-score> <sasv-score>, separated with a tabular ('\t') delimiter.

Example score.tsv

Approach A: Participant End2End SASV system

    spk     filename cm-score asv-score sasv-score
    E_2040  E_000001     -        -         0.001
    E_2040  E_000002     -        -         0.002
    ...

Approach B: Participant SASV system by combined participant CM and ASV systems

    spk     filename cm-score asv-score sasv-score
    E_2040  E_000001   0.001    0.003     0.005
    E_2040  E_000002   0.002    0.004     0.006
    ...

For any of the approaches above, participant should submit a single five-columns file. However, participant decides which approach suits his scenario and fill out corresponding columns. Fill out the column for missing system(-s) with dash symbols ('-').

For scores, please submit floating point values only (e.g., no NaNs / inf values).

You can find an example submissions at: Participate → Files → Download: Public Data
It is a zip archive of score files produced by our baseline systems.

Computational Notice

Please be patient while uploading your submission zip file. CodaLab does not show the progress bar, and it may take few minutes until submission is uploaded.
After submission is successful, status changes might take more than 5 minutes from Submitted to Finished.
Please track log files for warnings/error messages.

Please avoid submissions in the last minutes/hours of a phase.

Please find detailed results through downloading CodaLab outputs that are created during the scoring of your submission(s).

Notes

• You may update your submission within daily limits;
  these vary between phases of the ASVspoof 5 challenge.
• You may select one of your earlier submissions to be shown on the dashboard;
  if you wish so, just select the corresponding submission.
• Detailed results will be available after the progress stage; see Download output from scoring step, the output from your scoring step.

Rules and Disclaimer

Participants should refer to the ASVspoof 5 challenge evaluation plan for further details regarding the participation rules.