Hi, we confronted the problem on submitting "final test data". We got the following error messages:
============================================================================
Traceback (most recent call last):
File "/tmp/codalab/tmp9UObu2/run/program/evaluation.py", line 42, in
f1_A = f1_score(gt_labels_A, ans_labels_A, average="weighted")
File "/opt/conda/lib/python3.7/site-packages/sklearn/metrics/classification.py", line 1059, in f1_score
sample_weight=sample_weight)
File "/opt/conda/lib/python3.7/site-packages/sklearn/metrics/classification.py", line 1182, in fbeta_score
sample_weight=sample_weight)
File "/opt/conda/lib/python3.7/site-packages/sklearn/metrics/classification.py", line 1415, in precision_recall_fscore_support
pos_label)
File "/opt/conda/lib/python3.7/site-packages/sklearn/metrics/classification.py", line 1239, in _check_set_wise_labels
y_type, y_true, y_pred = _check_targets(y_true, y_pred)
File "/opt/conda/lib/python3.7/site-packages/sklearn/metrics/classification.py", line 71, in _check_targets
check_consistent_length(y_true, y_pred)
File "/opt/conda/lib/python3.7/site-packages/sklearn/utils/validation.py", line 205, in check_consistent_length
" samples: %r" % [int(l) for l in lengths])
ValueError: Found input variables with inconsistent numbers of samples: [9000, 45000]
====================================================================================
Was the latest test data not updated yet?
Posted by: Larry_Chen @ Dec. 1, 2024, 1:35 p.m.I am also facing the same issue.
Posted by: skmalviya @ Dec. 2, 2024, 2:42 p.m.Hey,
I apologize for the delayed response. Upon investigation, we found that the reference dataset was not updated to align with the new dataset. However, this issue has now been resolved.
Kindly check if everything is working as expected on your end. Additionally, we released a new dataset a few minutes ago—please use it for your predictions and submit your answers accordingly.
Hf dataset: https://huggingface.co/datasets/NasrinImp/Final_defactify_test_new
If you encounter any further issues, don’t hesitate to let us know by replying to this thread or emailing us directly.
Thank you for your patience and understanding!
Posted by: RajRoy1243 @ Dec. 2, 2024, 7:43 p.m.