Camera Splatting for Continuous View Optimization

2025-09-19

AI Summary

To address the challenge of modeling strong view-dependent phenomena—such as metallic specular highlights and fine-scale textures—in novel view synthesis, this paper introduces Camera Splatting. It is the first method to model cameras as differentiable 3D Gaussian distributions (“camera splats”) and deploy virtual point cameras near scene surfaces for continuous view sampling and optimization. Integrated within the 3D Gaussian Splatting framework, our approach jointly optimizes both the camera distribution parameters and the scene representation via differentiable rendering, enabling explicit, continuous modeling of view-dependent appearance. Experiments demonstrate that, compared to discrete sampling strategies like Farthest View Sampling, Camera Splatting achieves significantly improved reconstruction quality for complex view-dependent details—including metal reflections and textural patterns—while maintaining higher fidelity and consistency under extreme viewing angles.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Search and Optimization: Mixed Discrete/Continuous Search
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search

Abstract

We propose Camera Splatting, a novel view optimization framework for novel view synthesis. Each camera is modeled as a 3D Gaussian, referred to as a camera splat, and virtual cameras, termed point cameras, are placed at 3D points sampled near the surface to observe the distribution of camera splats. View optimization is achieved by continuously and differentiably refining the camera splats so that desirable target distributions are observed from the point cameras, in a manner similar to the original 3D Gaussian splatting. Compared to the Farthest View Sampling (FVS) approach, our optimized views demonstrate superior performance in capturing complex view-dependent phenomena, including intense metallic reflections and intricate textures such as text.

Problem

• Optimizing continuous novel view synthesis
• Modeling cameras as differentiable 3D Gaussians
• Capturing complex view-dependent reflections and textures

Innovation

• Camera Splatting for continuous view optimization
• Differentiably refining 3D Gaussian camera splats
• Point cameras sample near-surface for target distributions