The 4th Large-scale Video Object Segmentation - Track 3: Referring Video Object Segmentation Forum

Go back to competition Back to thread list Post in this thread

> Evaluation script fails

When I try to evaluate the result. It pops up an error "TypeError: a bytes-like object is required, not 'str'".
Does anyone meet the same problem?

Posted by: hoax @ Sept. 24, 2022, 12:47 a.m.

The full error messages are as follow:
WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
/tmp/codalab/tmp0txBIN/run/program/evaluate.py:71: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
annotation = annotation.astype(np.bool)
/tmp/codalab/tmp0txBIN/run/program/evaluate.py:72: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
segmentation = segmentation.astype(np.bool)
/tmp/codalab/tmp0txBIN/run/program/evaluate.py:27: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
seg = seg.astype(np.bool)
/tmp/codalab/tmp0txBIN/run/program/evaluate.py:71: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
annotation = annotation.astype(np.bool)
/tmp/codalab/tmp0txBIN/run/program/evaluate.py:72: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
segmentation = segmentation.astype(np.bool)
/tmp/codalab/tmp0txBIN/run/program/evaluate.py:27: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
seg = seg.astype(np.bool)
Traceback (most recent call last):
File "/tmp/codalab/tmp0txBIN/run/program/evaluate.py", line 240, in
output_file.write('Overall: {}\n'.format(overall))
TypeError: a bytes-like object is required, not 'str'

Posted by: hoax @ Sept. 24, 2022, 12:48 a.m.

I also met the same problem.

Posted by: MasterBin-IIAU @ Sept. 24, 2022, 8:17 a.m.

yes, me too

Posted by: xwt @ Sept. 24, 2022, 8:40 a.m.

Hi, guys. You can calculate the metrics based on the stdout.txt (click "View scoring output log") using the following script.
I have tested it with my previously submitted results.

def mean(obj):
return sum(obj)/len(obj)

if __name__ == "__main__":
J_dict, F_dict = {}, {}
with open("stdout.txt", "r") as f:
for line in f.readlines():
line = line.strip("\n").split(",")
key = line[0] + line[1]
J, F = float(line[-2]), float(line[-1])
if key not in J_dict:
J_dict[key] = []
if key not in F_dict:
F_dict[key] = []
J_dict[key].append(J)
F_dict[key].append(F)
J_list, F_list = [], []
for key in J_dict.keys():
J_list.append(mean(J_dict[key]))
F_list.append(mean(F_dict[key]))
J = mean(J_list)
F = mean(F_list)
print("J&F:", (J+F)/2)

Posted by: MasterBin-IIAU @ Sept. 24, 2022, 8:44 a.m.

Great, I just confirmed the above script works perfectly. Thanks!

Posted by: hoax @ Sept. 25, 2022, 7:49 p.m.

Sorry for the late reply. I just modified the scoring code to handle the error. Please let me know if you still have a problem.

Posted by: seonguk @ Sept. 28, 2022, 4:24 p.m.
Post in this thread