I was wondering if it was possible to use external libraries such as pickles or joblib to save the scaler object used to preprocess our data in order to apply the same preprocessing also to the data to compute the predictions.
I've seen that in the scikit-learn external libraries joblib should be included, but I've also seen that it's deprecated as it is better to import joblib directly. Would it be a problem in this competition or can we actually do it?
joblib is not included in the provided environment.
Posted by: an2dl.competitions @ Dec. 12, 2022, 9:15 a.m.Hello!
I'm having the same problem, I don't know how to load my saved scaler in the model.py file, I've tried to create one folder Model and another Scaler inside the SubmissionModel folder and implement the following code inside the function __init__:
self.model = tf.keras.models.load_model(os.path.join(path, 'SubmissionModel/Model'))
self.scaler = np.load(open(os.path.join(path, 'SubmissionModel/Scaler')), allow_pickle=True) #i used np.load because the libraries pickle and joblib are not allowed
However, it didn't work. I also tried different approaches (such as putting the model and the scaler in the same folder or creating a SubmissionScale folder) but it is still not working.
Is there any other way to do this? I don't know where to save the scaler file inside the .zip file
Posted by: pedrohgian @ Dec. 14, 2022, 11:17 p.m.