Hello guys.
after admission, I face these errors and I think these errors are because of the saving model in TensorFlow.
Does anyone know how I can fix it?
WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
2021-11-16 17:15:11.192001: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-11-16 17:15:12.358052: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 11483 MB memory: -> device: 0, name: GeForce GTX TITAN X, pci bus id: 0000:05:00.0, compute capability: 5.2
Traceback (most recent call last):
File "/multiverse/storage/lattari/Prj/phd/AN2DL/Competition_1_running_dir/worker_gpu1_dir/tmp/codalab/tmpIz_JDa/run/program/score.py", line 117, in <module>
M = model(submission_dir)
File "/multiverse/storage/lattari/Prj/phd/AN2DL/Competition_1_running_dir/worker_gpu1_dir/tmp/codalab/tmpIz_JDa/run/input/res/model.py", line 6, in __init__
self.model = tf.keras.models.load_model(os.path.join(path, 'SubmissionModel'))
File "/usr/local/lib/python3.6/dist-packages/keras/saving/save.py", line 205, in load_model
return saved_model_load.load(filepath, compile, options)
File "/usr/local/lib/python3.6/dist-packages/keras/saving/saved_model/load.py", line 140, in load
loaded = tf.__internal__.saved_model.load_partial(path, nodes_to_load, options=options)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/saved_model/load.py", line 769, in load_partial
return load_internal(export_dir, tags, options, filters=filters)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/saved_model/load.py", line 903, in load_internal
ckpt_options, options, filters)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/saved_model/load.py", line 165, in __init__
self._restore_checkpoint()
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/saved_model/load.py", line 477, in _restore_checkpoint
load_status.assert_existing_objects_matched()
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/tracking/util.py", line 853, in assert_existing_objects_matched
(list(unused_python_objects),))
AssertionError: Some Python objects were not bound to checkpointed values, likely due to changes in the Python program: [<tf.Variable 'Output/bias/v:0' shape=(14,) dtype=float32, numpy=