IGL-Nav: Incremental 3D Gaussian Localization for Image-goal Navigation

2025-08-01

AI Summary

This work addresses the image-goal navigation task by proposing a real-time localization framework based on incremental 3D Gaussian representations. To overcome the limitations of existing methods—namely, their inability to simultaneously achieve efficiency, accuracy, and online adaptability—the framework integrates monocular feedforward depth prediction, geometry-guided coarse localization, differentiable Gaussian rendering, and 6-DoF pose optimization, enabling dynamic scene reconstruction and continuous representation refinement during exploration. A key innovation lies in coupling 3D Gaussian graph optimization with discrete spatial matching, facilitating efficient 3D spatial alignment and precise pose estimation for target images captured from arbitrary viewpoints. Experiments demonstrate significant improvements over state-of-the-art methods across multiple benchmarks. The approach supports free-viewpoint navigation and has been successfully deployed on a real-world robotic platform, where navigation goals can be specified using a single image captured by a smartphone from any pose.

Technology Category

• Intelligent Robots: Localization, Mapping, and Navigation
• Computer Vision: 3D Computer Vision
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Visual navigation with an image as goal is a fundamental and challenging problem. Conventional methods either rely on end-to-end RL learning or modular-based policy with topological graph or BEV map as memory, which cannot fully model the geometric relationship between the explored 3D environment and the goal image. In order to efficiently and accurately localize the goal image in 3D space, we build our navigation system upon the renderable 3D gaussian (3DGS) representation. However, due to the computational intensity of 3DGS optimization and the large search space of 6-DoF camera pose, directly leveraging 3DGS for image localization during agent exploration process is prohibitively inefficient. To this end, we propose IGL-Nav, an Incremental 3D Gaussian Localization framework for efficient and 3D-aware image-goal navigation. Specifically, we incrementally update the scene representation as new images arrive with feed-forward monocular prediction. Then we coarsely localize the goal by leveraging the geometric information for discrete space matching, which can be equivalent to efficient 3D convolution. When the agent is close to the goal, we finally solve the fine target pose with optimization via differentiable rendering. The proposed IGL-Nav outperforms existing state-of-the-art methods by a large margin across diverse experimental configurations. It can also handle the more challenging free-view image-goal setting and be deployed on real-world robotic platform using a cellphone to capture goal image at arbitrary pose. Project page: https://gwxuan.github.io/IGL-Nav/.

Problem

• Efficient 3D localization of goal images in navigation
• Overcoming computational intensity in 3D Gaussian optimization
• Achieving accurate 6-DoF camera pose estimation during exploration

Innovation

• Incremental 3D Gaussian Localization framework
• Feed-forward monocular prediction updates
• Differentiable rendering for fine pose