MVP-Nav: Multi-layer Value Map Planner Navigator

2026-06-30

AI Summary

This work addresses the physical uncertainty and semantic-physical misalignment inherent in zero-shot object navigation using only RGB images, which lack explicit depth information. To overcome these challenges, the authors propose a physics-aware navigation framework that, for the first time, leverages a 3D foundation model to project 2D semantic instances into 3D oriented bounding boxes without depth input, thereby explicitly reconstructing physical occupancy and constructing a global spatial-semantic representation. Building upon this representation, a multi-layer value map is designed to jointly optimize high-level semantic priorities and low-level geometric constraints, enabling geometrically reliable path planning. Experiments demonstrate that the proposed method significantly outperforms existing RGB-only approaches on zero-shot navigation benchmarks, achieving state-of-the-art performance and confirming that structured physical priors can effectively compensate for the absence of active depth sensing.

Abstract

Zero-shot Object Goal Navigation (ZSON) with RGB-only perception poses a fundamental challenge for embodied agents, as the absence of explicit depth information introduces severe physical uncertainty and semantic-physical misalignment. Existing approaches either rely on high-level semantic reasoning without geometric grounding or learn end-to-end policies that lack explicit physical constraints, often resulting in semantically plausible but physically unsafe behaviors. In this paper, we propose MVP-Nav, a physical-aware RGB-only navigation framework that aligns perception, planning, and control with the real 3D world. MVP-Nav reconstructs explicit physical occupancy from monocular observations by leveraging 3D foundation models to project 2D semantic instances into 3D oriented bounding boxes, forming a global spatial semantic representation. To unify high-level semantic reasoning and low-level physical constraints, we introduce a Multi-layer Value Map (MVM) that integrates semantic priorities and reconstructed geometry into a shared cost space, enabling physically grounded geometric planning. Extensive experiments on zero-shot object navigation benchmarks demonstrate that MVP-Nav significantly outperforms existing depth-free methods, achieving state-of-the-art performance and validating that structured physical priors can effectively compensate for the absence of active depth sensors.

Problem

• Zero-shot Object Goal Navigation
• RGB-only perception
• physical uncertainty
• semantic-physical misalignment
• depth-free navigation

Innovation

• Zero-shot Object Goal Navigation
• RGB-only perception
• 3D foundation models
• Multi-layer Value Map
• physical-aware navigation