Hierarchical 3D Scene Graph Construction and Belief-based Planning for Semantic Navigation

2026-06-29

AI Summary

This work addresses the limitations of semantic navigation—namely, its reliance on partial observations, greedy decision-making, and poor efficiency over long distances—stemming from the absence of a structured global representation. To overcome these challenges, the authors propose a zero-shot navigation method based on a hierarchical 3D scene graph (HSG). The approach constructs a multi-granularity semantic topology online and, for the first time, employs HSG as an abstract global state representation. Integrated with a belief-driven hierarchical planning mechanism, it combines semantic priors with exploration evidence to simulate macro-actions and evaluate their long-term returns under limited visibility. Experiments in multiple high-fidelity simulation environments demonstrate that the method significantly outperforms current state-of-the-art approaches, achieving average improvements of 9.4% in success rate (SR) and 5.0% in success weighted by path length (SPL) on long-range tasks.

Abstract

Semantic navigation is a fundamental task for embodied agents operating in unseen environments, requiring both semantic understanding and long-term decision-making. Recent foundation models have empowered agents with rich semantic priors for this task. However, without structured global representations, decision-making often falls back on local observations and greedy strategies, resulting in inefficient exploration and myopic behaviors, especially in long-distance navigation. To address these challenges, we propose a zero-shot semantic navigation framework. Our method incrementally maintains an online Hierarchical 3D Scene Graph (HSG) to form a multi-granular semantic topology over objects, zones, and regions, serving as a compact state abstraction for global planning. Building on this memory, we introduce a hierarchical belief-based planning framework that fuses semantic priors with exploration evidence on the HSG, and performs finite-horizon rollouts on an HSG-based simulator to explicitly estimate the long-term expected returns of candidate macro-actions. This enables globally consistent decisions and reduces redundant backtracking. Extensive experiments in high-fidelity simulation environments across multiple tasks and datasets demonstrate that our method outperforms existing state-of-the-art methods, particularly in long-distance scenarios, where our approach improves SR and SPL by an average of 9.4\% and 5.0\%, respectively.

Problem

• semantic navigation
• hierarchical scene graph
• long-term planning
• embodied agents
• global representation

Innovation

• Hierarchical 3D Scene Graph
• belief-based planning
• semantic navigation
• zero-shot
• macro-action planning