Question about the shape of map_size.

[WangzcBruce]

The original dimension [8, 200, 200] could lead to errors. Should it be adjusted to [26, 200, 200]?
Reference link:

MagicDrive/configs/model/SDv1.5mv_rawbox.yaml

Line 31 in d2fad7d

map_size: [8, 200, 200]

python-BaseException
Error executing job with overrides: ['++runner.enable_xformers_memory_efficient_attention=false']
Traceback (most recent call last):
File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/hydra/_internal/utils.py", line 394, in _run_hydra
_run_app(
File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/hydra/_internal/utils.py", line 457, in _run_app
run_and_report(
File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/hydra/_internal/utils.py", line 222, in run_and_report
raise ex
File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/hydra/_internal/utils.py", line 219, in run_and_report
return func()
File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/hydra/_internal/utils.py", line 458, in
lambda: hydra.run(
File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 132, in run
_ = ret.return_value
File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/hydra/core/utils.py", line 260, in return_value
raise self._return_value
File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/hydra/core/utils.py", line 186, in run_job
ret.return_value = task_function(task_cfg)
File "/workspace/[email protected]/MagicDrive/tools/train.py", line 110, in main
runner.run()
File "/workspace/[email protected]/MagicDrive/magicdrive/runner/base_runner.py", line 343, in run
loss = self._train_one_stop(batch)
File "/workspace/[email protected]/MagicDrive/magicdrive/runner/multiview_runner.py", line 300, in _train_one_stop
model_pred = self.controlnet_unet(
File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/accelerate/utils/operations.py", line 553, in forward
return model_forward(*args, **kwargs)
File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/accelerate/utils/operations.py", line 541, in call
return convert_to_fp32(self.model_forward(*args, **kwargs))
File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/torch/autocast_mode.py", line 198, in decorate_autocast
return func(*args, **kwargs)
File "/workspace/[email protected]/MagicDrive/magicdrive/runner/multiview_runner.py", line 48, in forward
encoder_hidden_states_with_cam = self.controlnet(
File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/workspace/[email protected]/MagicDrive/magicdrive/networks/unet_addon_rawbox.py", line 848, in forward
controlnet_cond = self.controlnet_cond_embedding(controlnet_cond)
File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/workspace/[email protected]/MagicDrive/magicdrive/networks/map_embedder.py", line 67, in forward
embedding = self.conv_in(conditioning)
File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 446, in forward
return self._conv_forward(input, self.weight, self.bias)
File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 442, in _conv_forward
return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Given groups=1, weight of size [16, 8, 3, 3], expected input[18, 26, 200, 200] to have 8 channels, but got 26 channels instead
Steps: 0%| | 0/937700 [00:42<?, ?it/s]

[flymin]

No, it should be 8 in the latest version. We use raw_box for objects, so there is no object-related layers in the map.

[WangzcBruce]

Therefore, it is necessary to execute the command python tools/prepare_map_aux.py +process=train rather than utilizing the existing HD maps in order to match the hyperparameters-8 in the latest version?