Which dataset is best for apartments?

[trungphien]

Hi, @donydchen, I want to use MVSplat to create a virtual apartment tour application. Could you suggest some relevant datasets that I can use to fine-tune MVSplat? Thank you very much!

[donydchen]

Hi @trungphien, I think the ScanNet dataset might be one of the good options for your scenario. Cheers.

[trungphien]

But I only have a server with 8GB GPU for training. Would ScanNet be suitable in this case?

[donydchen]

Hey @trungphien, the 8G GPU seems too limited to train/fine-tune a better model. Below, I list some suggestions that might help reduce the memory consumption, but keep in mind that it will also decrease the performance to some extent.

• use a smaller patch_size for the swin attention, e.g., set multiview_trans_attn_split=4 in costvolume.yaml.
• reduce the transformer layers in backbone_multiview.py, e.g., set num_transformer_layers=4.
• remove the self-attn in the encoder multiview_transformer.py. Do NOT remove the cross-attn as it might significantly decrease the performance.
• replace all attention blocks with more memory-efficient implementation, e.g., xformers.ops.memory_efficient_attention.

Still, it might be much better to find a better GPU with a larger VRAM, e.g., 80G A100. Have fun tuning. Cheers.

[trungphien]

Thank you very much for your enthusiastic support!