Forums

VLSP2023 - ViVRC Challenge: Visual Reading Comprehension for Vietnamese Forum

Go back to competition Back to thread list Post in this thread

> Question about test set in the paper

I have a question about the dataset paper "OpenViVQA: Task, dataset, and multimodal fusion models for visual question answering in Vietnamese".
Is the current private test of the competition the same as the test set in the paper?

Thank you,
NguyenLe.

Posted by: NguyenLe @ Oct. 11, 2023, 10:50 a.m.

Of course, they are the same set except for some pseudo questions in the private test set to avoid manual annotation during this challenge.

Posted by: nhn_nlp_uit @ Oct. 11, 2023, 1:32 p.m.

I downloaded the private test from the competition and saw that there are 14000 questions. However, in the paper, the test set contains about 3500 questions.
Could you explain this difference?

Thank you very much.
NguyenLe.

Posted by: NguyenLe @ Oct. 11, 2023, 1:37 p.m.

There are pseudo-questions augmented in the test set to avoid manual annotation.

Posted by: nhn_nlp_uit @ Oct. 11, 2023, 10:12 p.m.

Will the original test set in the paper be available during the post-evaluation phase?

Posted by: NguyenLe @ Oct. 12, 2023, 3:41 a.m.

The OpenViVQA dataset will be publicly available after this challenge.

Posted by: nhn_nlp_uit @ Oct. 12, 2023, 7:33 a.m.
Post in this thread