Surf-NeRF: Surface Regularised Neural Radiance Fields

2024-11-27

arXiv.org

AI Summary

NeRF suffers from geometry-radiance ambiguity, hindering physically consistent and accurate geometric reconstruction. To address this, we propose a surface light field curriculum learning framework that explicitly decouples geometry from view-dependent appearance via four physics-driven regularizations: normal consistency, geometric smoothness, and Lambertian/specular reflectance separation. This constitutes the first explicit geometry-appearance disentanglement in radiance fields. Our method integrates curriculum learning, surface light field modeling, multi-scale positional encoding, and a hash grid NeRF architecture. Experiments demonstrate significant improvements in normal estimation accuracy—+14.4% with positional encoding and +9.2% with hash grids—substantially enhancing geometric fidelity. The approach is compatible with mainstream NeRF variants and robustly supports geometry-sensitive downstream tasks, including 3D reconstruction and novel-view synthesis.

Technology Category

• Machine Learning: Learning with Manifolds
• Computer Vision: Low Level & Physics-based Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Neural Radiance Fields (NeRFs) provide a high fidelity, continuous scene representation that can realistically represent complex behaviour of light. Despite recent works like Ref-NeRF improving geometry through physics-inspired models, the ability for a NeRF to overcome shape-radiance ambiguity and converge to a representation consistent with real geometry remains limited. We demonstrate how curriculum learning of a surface light field model helps a NeRF converge towards a more geometrically accurate scene representation. We introduce four additional regularisation terms to impose geometric smoothness, consistency of normals and a separation of Lambertian and specular appearance at geometry in the scene, conforming to physical models. Our approach yields improvements of 14.4% to normals on positionally encoded NeRFs and 9.2% on grid-based models compared to current reflection-based NeRF variants. This includes a separated view-dependent appearance, conditioning a NeRF to have a geometric representation consistent with the captured scene. We demonstrate compatibility of our method with existing NeRF variants, as a key step in enabling radiance-based representations for geometry critical applications.

Problem

• Overcoming shape-radiance ambiguity in NeRF for accurate geometry
• Enhancing geometric smoothness and normal consistency in scene representation
• Separating Lambertian and specular appearance for realistic light modeling

Innovation

• Curriculum learning of surface light field model
• Lattice-based hash encoding for geometric accuracy
• Four regularization terms for physical consistency