Speaker Attribution 2023 Competition Forum

Go back to competition Back to thread list Post in this thread

> Error in evaluation

Dear organizers,

we receive an error when submitting a zip file:

WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
Traceback (most recent call last):
File "/tmp/codalab/tmpOoepZd/run/program/score.py", line 262, in
x.eval(args.input_dir, args.output_dir)
File "/tmp/codalab/tmpOoepZd/run/program/score.py", line 159, in eval
e = st(system_ann, gold_ann, doc2sentid2string_map)
File "/tmp/codalab/tmpOoepZd/run/program/classes.py", line 1183, in __init__
self.add_eval(EvaluateSubtrack1(annotator_cas, gold_cas, doc2sentid2string_map, **kwargs),
File "/tmp/codalab/tmpOoepZd/run/program/classes.py", line 769, in __init__
assert my_key not in match_gold_2_sys_cues
AssertionError

To our knowledge, the data format of the json files in the zip file is correct. Could you take a look at this error?

With kind regards,
Team CPAa

Posted by: nesasio93 @ July 24, 2023, 8:24 a.m.

Dear Ines,

ids 16 to 19 with experiment_19.zip as well as id 20 with said zip file are failing.

With kind regards,
Team CPAa

Posted by: nesasio93 @ July 24, 2023, 10:10 a.m.
Post in this thread