Dear participants,
We have made some changes in the evaluation script. Now when an answer is not present in the corresponding article, it will count as 0 for exact match and F1 overlap, but it will not prevent the evaluation of the rest of the submission.
On top of this, you can now see in the error log (as warnings) all the answers that did not belong to the article text, so you can check them and make the corrections you need.
We have re-run the script for the latest submissions of all participants, but you can still resubmit or send new submissions during this phase.
Regards,
QuALES organizers