Expanding to Autonomous Driving Datasets

[wendy0527]

Hello, I would like to know if it's possible to extend MVSplat to autonomous driving datasets like nuScenes, where there are multiple camera images at each moment with a small overlapping area between each pair. If so, would you recommend training the model with: all camera images from each moment as the context input, or only the overlapping images between each pair of cameras?

[donydchen]

Hi @wendy0527, I do not yet have experience working with nuScenes. But you are welcome to try our MVSplat on any related new datasets, and we are more than happy to help. I would suggest you only use image pairs with overlapping regions since image pairs with no overlap might harm the effectiveness of cost volume.

[Mr-ind1fferent]

Hello, I would like to know if it's possible to extend MVSplat to autonomous driving datasets like nuScenes, where there are multiple camera images at each moment with a small overlapping area between each pair. If so, would you recommend training the model with: all camera images from each moment as the context input, or only the overlapping images between each pair of cameras?

Have you tried successfully?