Drive-Through 3D Vehicle Exterior Reconstruction via Dynamic-Scene SfM and Distortion-Aware Gaussian Splatting

2026-03-27

AI Summary

This work addresses the challenges of high-fidelity 3D reconstruction of moving vehicles in automotive dealership walkthrough scenarios, where dynamic foregrounds, background clutter, wide-angle lens distortion, specular reflections from car paint, and non-rigid wheel motion complicate the process. The authors propose an end-to-end reconstruction pipeline that leverages a dual-pole camera system combined with SAM 3 instance segmentation and motion gating to isolate the target vehicle. A semantic-guided RoMa v2 matcher extracts robust features directly from undistorted images, while CAD-informed rig-aware structure-from-motion (SfM) refines camera poses. Furthermore, a distortion-aware 3D Gaussian Splatting framework (3DGUT) integrated with an MCMC-based densification strategy enables photorealistic rendering of reflective surfaces. The method achieves dealership-scale, studio-free vehicle reconstruction, yielding interactive, inspection-grade 3D models with a PSNR of 28.66 dB on a real-world dataset of 25 vehicles—outperforming standard 3D Gaussian Splatting by 3.85 dB.

Abstract

High-fidelity 3D reconstruction of vehicle exteriors improves buyer confidence in online automotive marketplaces, but generating these models in cluttered dealership drive-throughs presents severe technical challenges. Unlike static-scene photogrammetry, this setting features a dynamic vehicle moving against heavily cluttered, static backgrounds. This problem is further compounded by wide-angle lens distortion, specular automotive paint, and non-rigid wheel rotations that violate classical epipolar constraints. We propose an end-to-end pipeline utilizing a two-pillar camera rig. First, we resolve dynamic-scene ambiguities by coupling SAM 3 for instance segmentation with motion-gating to cleanly isolate the moving vehicle, explicitly masking out non-rigid wheels to enforce strict epipolar geometry. Second, we extract robust correspondences directly on raw, distorted 4K imagery using the RoMa v2 learned matcher guided by semantic confidence masks. Third, these matches are integrated into a rig-aware SfM optimization that utilizes CAD-derived relative pose priors to eliminate scale drift. Finally, we use a distortion-aware 3D Gaussian Splatting framework (3DGUT) coupled with a stochastic Markov Chain Monte Carlo (MCMC) densification strategy to render reflective surfaces. Evaluations on 25 real-world vehicles across 10 dealerships demonstrate that our full pipeline achieves a PSNR of 28.66 dB, an SSIM of 0.89, and an LPIPS of 0.21 on held-out views, representing a 3.85 dB improvement over standard 3D-GS, delivering inspection-grade interactive 3D models without controlled studio infrastructure.

Problem

• 3D reconstruction
• dynamic scene
• vehicle exterior
• lens distortion
• specular reflection

Innovation

• Dynamic-Scene SfM
• Distortion-Aware Gaussian Splatting
• Motion-Gated Instance Segmentation
• Rig-Aware Structure-from-Motion
• MCMC Densification