BRUM: Robust 3D Vehicle Reconstruction from 360 Sparse Images

2025-07-16

AI Summary

This paper addresses the low accuracy and poor robustness of vehicle 3D reconstruction from sparse views. To this end, we propose an end-to-end Gaussian point-based rendering framework. Methodologically: (1) DUSt3R replaces conventional SfM for more robust sparse pose estimation; (2) depth-aware Gaussian point rendering is introduced, coupled with a selective photometric loss that enforces constraints only on high-confidence pixels, thereby improving geometric consistency; (3) multi-view synthesis is integrated with depth-map guidance to enhance generalization to real-world scenes. Our method achieves state-of-the-art performance across multiple vehicle reconstruction benchmarks, significantly outperforming existing approaches. Extensive experiments on both synthetic and real-world bus datasets demonstrate strong adaptability to extremely sparse inputs (e.g., only 3–5 views) and deliver high-fidelity reconstructions. The framework exhibits superior robustness, geometric fidelity, and cross-domain generalization—particularly under challenging sparse-view conditions typical in automotive applications.

Technology Category

• Computer Vision: 3D Computer Vision
• Intelligent Robots: State Estimation
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Security and Privacy: Large-scale security measurements

Abstract

Accurate 3D reconstruction of vehicles is vital for applications such as vehicle inspection, predictive maintenance, and urban planning. Existing methods like Neural Radiance Fields and Gaussian Splatting have shown impressive results but remain limited by their reliance on dense input views, which hinders real-world applicability. This paper addresses the challenge of reconstructing vehicles from sparse-view inputs, leveraging depth maps and a robust pose estimation architecture to synthesize novel views and augment training data. Specifically, we enhance Gaussian Splatting by integrating a selective photometric loss, applied only to high-confidence pixels, and replacing standard Structure-from-Motion pipelines with the DUSt3R architecture to improve camera pose estimation. Furthermore, we present a novel dataset featuring both synthetic and real-world public transportation vehicles, enabling extensive evaluation of our approach. Experimental results demonstrate state-of-the-art performance across multiple benchmarks, showcasing the method's ability to achieve high-quality reconstructions even under constrained input conditions.

Problem

• Reconstructing 3D vehicles from sparse-view inputs
• Improving pose estimation and photometric loss for accuracy
• Enhancing Gaussian Splatting with synthetic and real-world data

Innovation

• Enhances Gaussian Splatting with selective photometric loss
• Uses DUSt3R for robust camera pose estimation
• Integrates depth maps to synthesize novel views