Hi,
When using the baseline model I got an error at the "Train a new detector" script:
RuntimeError: ReDet: ReResNet: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument mat2 in method wrapper_bmm)
What should I change in the script?
Thanks
Posted by: roigu @ Feb. 8, 2023, 2:32 p.m.Hi,
We suspect it might have something to do with the installed packages dependencies. Please make sure you have installed all the required packages with the correct version.
Let us know if that helped,
Neta
MAFAT Challenge Team
encountering the same error
Posted by: gfreund @ Feb. 10, 2023, 12:42 p.m.Hi,
We encountered this problem under two circumstances.
The first is when pytorch version is above 1.12. the correct package version is installed under the installation section in the notebook.
The second is when trying to re-run the training cell more than once. for that we do not have a solution other than restarting the notebook.
Shai,
MAFAT Challenge Team
Hi, I got this error:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-23-60ff7d29004e> in <module>
35 # Create work_dir
36 mmcv.mkdir_or_exist(osp.abspath(cfg.work_dir))
---> 37 train_detector(model, datasets, cfg, distributed=False, validate=False)
11 frames
/usr/local/lib/python3.9/dist-packages/mmdet/apis/train.py in train_detector(model, dataset, cfg, distributed, validate, timestamp, meta)
244 elif cfg.load_from:
245 runner.load_checkpoint(cfg.load_from)
--> 246 runner.run(data_loaders, cfg.workflow)
/usr/local/lib/python3.9/dist-packages/mmcv/runner/epoch_based_runner.py in run(self, data_loaders, workflow, max_epochs, **kwargs)
134 if mode == 'train' and self.epoch >= self._max_epochs:
135 break
--> 136 epoch_runner(data_loaders[i], **kwargs)
137
138 time.sleep(1) # wait for some hooks like loggers to finish
/usr/local/lib/python3.9/dist-packages/mmcv/runner/epoch_based_runner.py in train(self, data_loader, **kwargs)
47 self.call_hook('before_train_epoch')
48 time.sleep(2) # Prevent possible deadlock during epoch transition
---> 49 for i, data_batch in enumerate(self.data_loader):
50 self.data_batch = data_batch
51 self._inner_iter = i
/usr/local/lib/python3.9/dist-packages/torch/utils/data/dataloader.py in __iter__(self)
442 return self._iterator
443 else:
--> 444 return self._get_iterator()
445
446 @property
/usr/local/lib/python3.9/dist-packages/torch/utils/data/dataloader.py in _get_iterator(self)
388 else:
389 self.check_worker_number_rationality()
--> 390 return _MultiProcessingDataLoaderIter(self)
391
392 @property
/usr/local/lib/python3.9/dist-packages/torch/utils/data/dataloader.py in __init__(self, loader)
1113 _utils.signal_handling._set_SIGCHLD_handler()
1114 self._worker_pids_set = True
-> 1115 self._reset(loader, first_iter=True)
1116
1117 def _reset(self, loader, first_iter=False):
/usr/local/lib/python3.9/dist-packages/torch/utils/data/dataloader.py in _reset(self, loader, first_iter)
1146 # prime the prefetch loop
1147 for _ in range(self._prefetch_factor * self._num_workers):
-> 1148 self._try_put_index()
1149
1150 def _try_get_data(self, timeout=_utils.MP_STATUS_CHECK_INTERVAL):
/usr/local/lib/python3.9/dist-packages/torch/utils/data/dataloader.py in _try_put_index(self)
1380
1381 try:
-> 1382 index = self._next_index()
1383 except StopIteration:
1384 return
/usr/local/lib/python3.9/dist-packages/torch/utils/data/dataloader.py in _next_index(self)
669
670 def _next_index(self):
--> 671 return next(self._sampler_iter) # may raise StopIteration
672
673 def _next_data(self):
/usr/local/lib/python3.9/dist-packages/torch/utils/data/sampler.py in __iter__(self)
245 batch = [0] * self.batch_size
246 idx_in_batch = 0
--> 247 for idx in self.sampler:
248 batch[idx_in_batch] = idx
249 idx_in_batch += 1
/usr/local/lib/python3.9/dist-packages/mmdet/datasets/samplers/group_sampler.py in __iter__(self)
34 [indice, np.random.choice(indice, num_extra)])
35 indices.append(indice)
---> 36 indices = np.concatenate(indices)
37 indices = [
38 indices[i * self.samples_per_gpu:(i + 1) * self.samples_per_gpu]
/usr/local/lib/python3.9/dist-packages/numpy/core/overrides.py in concatenate(*args, **kwargs)
ValueError: need at least one array to concatenate
Posted by: rakhaja @ March 28, 2023, 2:02 a.m.Hi,
I assume the error occurred because the detector didn't recognize the images, so please ensure the following:
1. You wrote the right path, and it is possible to read files from there.
2. The image is not a 'nonetype' object (The "cv2.imread" function does not throw any error if the filename is wrong).
Let us know if the issue is resolved.
Neta,
MAFAT Challenge Team