Hello,
I downloaded the new starting_kit.zip (22.02.2022 19:30 CET), updated the files and ran my existing agent.py which was running fine until yesterday. The average ALC and final score came to be 0.0 (which was not the case till yesterday) and only 45/100 png files were generated in the output visualization folder.
The occasional error that was being caught was a KeyError emerging from the try block starting from L374 in score.py when scoring the runs.
The problem:
When reading the .csv files on L376m the algo_index read is of type float by default. When reading the learning curve inside reveal_score_on_the_test_set(), the keys are all integers represented as strings for the algorithms.
Solution 1:
L376 can be updated to be,
df = pd.read_csv(output_file, names=['algo_index', 'algo_time_spent', 'total_time_spent', 'score'], dtype={"algo_index": str})
OR
Solution 2:
L199 can be updated to be,
lc = env.test_learning_curves[dataset_name][str(int(algo_index))]
Performing either of these 2 changes fixes the execution of score.py without making changes to anything else.
In case, I am using an old version of the starting_kit, kindly pardon me and point me to the correct .zip file.
Also, if the environment code errors locally, I am sceptical to submit solutions and run down my submission quota. It would be nice if you could kindly look into this and tell us the best recourse.
Thank you!
Posted by: automl-freiburg @ Feb. 22, 2022, 7:42 p.m.Hello,
I updated the starter kit by downloading again (23.02.2022 09:30 CET). It fixes the issue I reported and I notice the change in the type enforcing when reading the CSV into Pandas.
Many thanks for the prompt update!
Thank you.
Posted by: automl-freiburg @ Feb. 23, 2022, 9:35 a.m.Hi,
I checked and I couldn’t reproduce your error.
I printed the type of the algo_index variable in line 198 by using: print(type(algo_index))
And it gave me the str type: <class 'str’> (not float as you had)
However, to make sure the error will not happen, I updated the starting kit to explicitly specify the data types when loading the csv files.
Thank you for reporting this. If you have any further questions please let us know.
Best regards,
Hung
Hello Hung,
Thanks for the prompt action nonetheless. Appreciate it!