DoF-Gaussian: Controllable Depth-of-Field for 3D Gaussian Splatting

2025-03-02

AI Summary

Existing 3D Gaussian Splatting (3D-GS) assumes pinhole, fully focused imaging, limiting its ability to model real-world shallow depth-of-field (DoF) effects. To address this, we propose the first controllable-DoF extension of 3D-GS. Methodologically, we (i) integrate a geometric optics lens model—incorporating aperture, focal length, and object distance—into the 3D-GS rendering pipeline, enabling physically consistent dynamic focus and defocus synthesis; (ii) introduce a scene-adaptive depth prior and a defocus-focus alignment mechanism to improve refocusing accuracy; and (iii) construct the first synthetic dataset explicitly supporting DoF-aware refocusing. Experiments demonstrate that our approach preserves real-time rendering performance while significantly enhancing focus localization accuracy. It enables interactive focus adjustment and precise control over blur intensity, advancing 3D-GS toward realistic camera imaging modeling.

Technology Category

• Computer Vision: 3D Computer Vision
• Search and Optimization: Sampling/Simulation-based Search
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Recent advances in 3D Gaussian Splatting (3D-GS) have shown remarkable success in representing 3D scenes and generating high-quality, novel views in real-time. However, 3D-GS and its variants assume that input images are captured based on pinhole imaging and are fully in focus. This assumption limits their applicability, as real-world images often feature shallow depth-of-field (DoF). In this paper, we introduce DoF-Gaussian, a controllable depth-of-field method for 3D-GS. We develop a lens-based imaging model based on geometric optics principles to control DoF effects. To ensure accurate scene geometry, we incorporate depth priors adjusted per scene, and we apply defocus-to-focus adaptation to minimize the gap in the circle of confusion. We also introduce a synthetic dataset to assess refocusing capabilities and the model's ability to learn precise lens parameters. Our framework is customizable and supports various interactive applications. Extensive experiments confirm the effectiveness of our method. Our project is available at https://dof-gaussian.github.io.

Problem

• Extends 3D Gaussian Splatting to handle shallow depth-of-field images.
• Develops a lens-based model to control depth-of-field effects.
• Introduces a synthetic dataset for evaluating refocusing capabilities.

Innovation

• Lens-based imaging model for DoF control
• Depth priors adjusted per scene
• Defocus-to-focus adaptation for accuracy