Dear Organisers,
My submission has 499 rows (this is for English), but still I get the below given error. We tested our file with evaluation script provided by you by giving both gold and prediction files as the prediction file itself and we got SAS and exact as 1.00 (since both gold and pred are same files). So what is formatting issue we are facing kindly help us.
regards
VSP_CLRG Team
-- Error Log --
INFO | Sub: "9084";"Supporting the Company
INFO | Sub: "9087";"Finally, we pledged to
INFO | Sub: "9088.a";"Following Savills In
INFO | Sub: "9088.b";"Following Savills In
INFO | * Loading reference data
INFO | * Loading prediction data
INFO | Load Data: check data set length = False
INFO | Load Data: check data set ref. context = False
Traceback (most recent call last):
File "/tmp/codalab/tmpGTJ9_o/run/program/task_evaluate.py", line 256, in
main()
File "/tmp/codalab/tmpGTJ9_o/run/program/task_evaluate.py", line 249, in main
args.func(args)
File "/tmp/codalab/tmpGTJ9_o/run/program/task_evaluate.py", line 218, in from_folder
evaluate_files(g_file, s_file, o_file)
File "/tmp/codalab/tmpGTJ9_o/run/program/task_evaluate.py", line 175, in evaluate_files
assert len(y_true) == len(y_pred), f"{len(y_true)} / {len(y_pred)}"
AssertionError: 499 / 1
The problem can be caused because the label of the first column is Index and must be ID.
Posted by: lli-uam @ Nov. 5, 2024, 4:03 p.m.Do not quote the identifiers in the first column and check the first row to contain ID;Text;Question;Answer.
Posted by: lli-uam @ Nov. 5, 2024, 5:35 p.m.What does "do not quote identifier" mean here?
I also get assertion error on num of rows, off by 1.
Can we have better error feedback, as most of my submissions fail and I do not know which aspect of the answer caused the failure.
Posted by: nirvanatear @ Nov. 6, 2024, 4:11 a.m.I think we have resolved your problem by e-mail, but for those of you with the same problem, what I mentioned above is that the values in the ID column should not be in quotation marks. In other words, 12345 is valid, but "12345" is not.
Posted by: lli-uam @ Nov. 6, 2024, 8:15 p.m.