Parameter-Free Neural Lens Blur Rendering for High-Fidelity Composites

2025-11-21

AI Summary

Existing mixed-reality lens blur synthesis methods rely on camera parameters (e.g., focal length, aperture) and scene depth estimation to compute the circle of confusion (CoC), limiting accessibility and generalizability for non-expert users. This paper proposes the first end-to-end lens blur synthesis framework that requires neither camera parameters nor explicit depth input. Our method first introduces a lightweight CoC estimation network that directly predicts a spatially varying, symbolic CoC map from a single RGB image. It then establishes a linear mapping between the estimated CoC and virtual object depth, integrated with a neural refocusing renderer to synthesize high-fidelity bokeh effects. By jointly leveraging image semantic features and virtual geometric cues, our approach achieves natural depth transitions and photorealistic real-virtual fusion. Extensive quantitative and qualitative evaluations on multiple benchmarks demonstrate consistent superiority over state-of-the-art methods, significantly enhancing practicality and generality for real-world MR applications.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Consistent and natural camera lens blur is important for seamlessly blending 3D virtual objects into photographed real-scenes. Since lens blur typically varies with scene depth, the placement of virtual objects and their corresponding blur levels significantly affect the visual fidelity of mixed reality compositions. Existing pipelines often rely on camera parameters (e.g., focal length, focus distance, aperture size) and scene depth to compute the circle of confusion (CoC) for realistic lens blur rendering. However, such information is often unavailable to ordinary users, limiting the accessibility and generalizability of these methods. In this work, we propose a novel compositing approach that directly estimates the CoC map from RGB images, bypassing the need for scene depth or camera metadata. The CoC values for virtual objects are inferred through a linear relationship between its signed CoC map and depth, and realistic lens blur is rendered using a neural reblurring network. Our method provides flexible and practical solution for real-world applications. Experimental results demonstrate that our method achieves high-fidelity compositing with realistic defocus effects, outperforming state-of-the-art techniques in both qualitative and quantitative evaluations.

Problem

• Estimating lens blur without camera parameters
• Rendering realistic defocus for virtual composites
• Eliminating dependency on depth and metadata

Innovation

• Estimates CoC map directly from RGB images
• Infers CoC values via linear depth relationship
• Renders lens blur using neural reblurring network