LookCloser: Frequency-aware Radiance Field for Tiny-Detail Scene

2025-03-24

AI Summary

NeRF modeling often suffers from frequency-domain imbalance, making it difficult to simultaneously preserve global structure and fine local details. To address this, we propose FA-NeRF—a single-model, multi-scale representation framework based on frequency-domain expansion. FA-NeRF introduces the first 3D frequency quantization method and a frequency-aware rendering mechanism, incorporating a differentiable frequency query grid and a frequency-adaptive feature reweighting strategy to enable hierarchical spectral analysis and efficient feature modulation. Evaluated on multiple challenging real-world datasets, FA-NeRF achieves PSNR gains exceeding 2.1 dB over state-of-the-art methods including Instant-NGP and TensoRF. It significantly improves joint fidelity of global geometric consistency and high-frequency texture details. By unifying multi-scale representation within a coherent frequency-domain paradigm, FA-NeRF establishes a principled, efficient foundation for immersive scene reconstruction.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Humans perceive and comprehend their surroundings through information spanning multiple frequencies. In immersive scenes, people naturally scan their environment to grasp its overall structure while examining fine details of objects that capture their attention. However, current NeRF frameworks primarily focus on modeling either high-frequency local views or the broad structure of scenes with low-frequency information, which is limited to balancing both. We introduce FA-NeRF, a novel frequency-aware framework for view synthesis that simultaneously captures the overall scene structure and high-definition details within a single NeRF model. To achieve this, we propose a 3D frequency quantification method that analyzes the scene's frequency distribution, enabling frequency-aware rendering. Our framework incorporates a frequency grid for fast convergence and querying, a frequency-aware feature re-weighting strategy to balance features across different frequency contents. Extensive experiments show that our method significantly outperforms existing approaches in modeling entire scenes while preserving fine details.

Problem

• Balancing high and low frequency scene modeling
• Simultaneously capturing scene structure and fine details
• Analyzing frequency distribution for better rendering

Innovation

• Frequency-aware NeRF for detail and structure
• 3D frequency quantification for scene analysis
• Frequency grid and feature re-weighting strategy