EAGOR: Embodied Reasoning in Omni-direction

2026-07-07

AI Summary

Existing vision-language models neglect the spherical geometry of 360° panoramic images, leading to inconsistent directional estimation under agent motion and hindering embodied tasks such as mapless navigation. This work proposes EAGOR, a framework that, without requiring training, formulates directional reasoning as recursive Bayesian estimation on the sphere. It introduces a Spherical Harmonic Belief Field (SH-BF) to construct a global, rotation-aware directional representation directly on the spherical manifold, thereby avoiding seam artifacts, distortions, and interpolation errors inherent in equirectangular projection (ERP). Coupled with equivariant belief propagation, the method achieves geometrically consistent inference. Evaluated on HOS and OSR-Bench, EAGOR yields relative improvements of 34.4% and 45.6%, respectively, along with a 14.6% increase in navigation success rate, 17.7% fewer steps, and a 24.5% reduction in average angular error.

Abstract

Omni-directional (360°) cameras provide embodied agents with a holistic view of their surroundings, making them suited for directional reasoning in tasks such as navigation and object search. Existing Vision Language Models (VLMs) project 360° observations to 2D equirectangular projection (ERP) images and process them using architectures designed for perspective images. However, they ignore the spherical nature of 360° observations, where each pixel represents a viewing direction relative to the agent. Consequently, their direction estimates often become inconsistent under camera view transformations caused by agent motion. This limitation is particularly critical for map-free navigation, where the agent must continuously estimate the target direction in its egocentric frame. We propose EAGOR, a training-free, geometry-aware framework for embodied 360° directional reasoning. Instead of predicting target directions as ERP image coordinates, EAGOR formulates directional reasoning as recursive Bayesian estimation directly on the sphere. It maintains a continuous belief over target directions and propagates it equivariantly under agent motion without training the backbone VLMs. To achieve this, we introduce the Spherical Harmonic Belief Field (SH-BF), whose spherical harmonic representation provides a globally defined, rotation-aware basis for directional estimation on the spherical manifold. This formulation eliminates ERP seam discontinuities, latitude distortions, and interpolation errors. We evaluate EAGOR on two benchmark datasets and real-world experiments with a legged robot across directional reasoning tasks. EAGOR consistently outperforms existing methods, achieving average relative gains of +34.4% and +45.6% on HOS and OSR-Bench, respectively, while improving navigation success by +14.6%, reducing step count by 17.7%, and lowering mean angular error by 24.5%.

Problem

• omnidirectional vision
• directional reasoning
• spherical representation
• embodied navigation
• view transformation inconsistency

Innovation

• Embodied Reasoning
• 360-degree Vision
• Spherical Harmonics
• Recursive Bayesian Estimation
• Equivariant Propagation