🤖 AI Summary
Existing single-step feedforward networks struggle to regress static Gaussian primitives suitable for all viewing angles in novel view synthesis from pose-free images, limiting reconstruction fidelity. This work proposes a viewpoint-adaptive dynamic Gaussian splatting method that transforms static representations into a view-aware dynamic splatting mechanism. Specifically, a lightweight dynamic MLP predicts residual updates to Gaussian attributes—including position, scale, rotation, opacity, and color—conditioned on the target viewpoint coordinates. By adapting Gaussian parameters dynamically to each novel view, the method significantly enhances synthesis quality while maintaining high computational efficiency, achieving state-of-the-art fidelity with inference at 17 FPS and real-time rendering at 154 FPS.
📝 Abstract
We present ViewSplat, a view-adaptive 3D Gaussian splatting network for novel view synthesis from unposed images. While recent feed-forward 3D Gaussian splatting has significantly accelerated 3D scene reconstruction by bypassing per-scene optimization, a fundamental fidelity gap remains. We attribute this bottleneck to the limited capacity of single-step feed-forward networks to regress static Gaussian primitives that satisfy all viewpoints. To address this limitation, we shift the paradigm from static primitive regression to view-adaptive dynamic splatting. Instead of a rigid Gaussian representation, our pipeline learns a view-adaptable latent representation. Specifically, ViewSplat initially predicts base Gaussian primitives alongside the weights of dynamic MLPs. During rendering, these MLPs take target view coordinates as input and predict view-dependent residual updates for each Gaussian attribute (i.e., 3D position, scale, rotation, opacity, and color). This mechanism, which we term view-adaptive dynamic splatting, allows each primitive to rectify initial estimation errors, effectively capturing high-fidelity appearances. Extensive experiments demonstrate that ViewSplat achieves state-of-the-art fidelity while maintaining fast inference (17 FPS) and real-time rendering (154 FPS).