NoPe-NeRF++: Local-to-Global Optimization of NeRF with No Pose Prior

2025-11-21

AI Summary

Existing methods (e.g., NoPe-NeRF) model only local image relationships, limiting their ability to accurately recover camera poses in complex scenes without pose priors. To address this, we propose a local-to-global NeRF pose optimization framework: (1) relative pose initialization via feature trajectories; (2) joint optimization of local radiance fields and camera poses; and (3) explicit feature-matching constraints to drive geometrically consistent global bundle adjustment. This is the first work to seamlessly integrate local appearance representations with global geometric cues within NeRF, enabling end-to-end, pose-prior-free robust reconstruction. Extensive experiments on multiple benchmarks demonstrate significant improvements in camera pose estimation accuracy and novel-view synthesis fidelity—particularly under challenging conditions including dynamic occlusions, textureless regions, and large-baseline configurations.

Technology Category

• Search and Optimization: Local Search
• Computer Vision: Learning & Optimization for CV
• Intelligent Robots: State Estimation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

In this paper, we introduce NoPe-NeRF++, a novel local-to-global optimization algorithm for training Neural Radiance Fields (NeRF) without requiring pose priors. Existing methods, particularly NoPe-NeRF, which focus solely on the local relationships within images, often struggle to recover accurate camera poses in complex scenarios. To overcome the challenges, our approach begins with a relative pose initialization with explicit feature matching, followed by a local joint optimization to enhance the pose estimation for training a more robust NeRF representation. This method significantly improves the quality of initial poses. Additionally, we introduce global optimization phase that incorporates geometric consistency constraints through bundle adjustment, which integrates feature trajectories to further refine poses and collectively boost the quality of NeRF. Notably, our method is the first work that seamlessly combines the local and global cues with NeRF, and outperforms state-of-the-art methods in both pose estimation accuracy and novel view synthesis. Extensive evaluations on benchmark datasets demonstrate our superior performance and robustness, even in challenging scenes, thus validating our design choices.

Problem

• Optimizing Neural Radiance Fields without requiring camera pose priors
• Improving pose estimation accuracy in complex scenarios using local-to-global optimization
• Enhancing novel view synthesis quality through geometric consistency constraints

Innovation

• Local-to-global optimization without pose priors
• Relative pose initialization with feature matching
• Bundle adjustment with geometric consistency constraints