GroundControl: Anticipating Navigation Failures in Vision-Language Agents via Trajectory-Consistent Uncertainty Estimates

2026-06-18

AI Summary

This work addresses the challenge that vision-language navigation agents often fail due to oscillation, stagnation, or inefficient detours, yet lack prospective uncertainty signals to anticipate such trajectory-level anomalies. The authors propose a trajectory consistency-based uncertainty estimation method that models the dynamic evolution of target distance using a constant-velocity Kalman filter and integrates multi-dimensional trajectory features—including normalized innovation statistics, path efficiency, and oscillatory behavior—to enable early detection of anomalous navigation. The study introduces the SRCN protocol and employs AURC/E-AURC metrics for standalone evaluation of uncertainty quality. Evaluated across five EB-Navigation datasets, the approach significantly outperforms existing baselines, achieving an E-AURC_SR of 0.0024 with GPT-4o—approaching oracle-level performance.

Abstract

Vision-language navigation agents achieve competitive average success on benchmark tasks, yet failures often arise through predictable trajectory-level breakdowns such as oscillation, stagnation, or inefficient detours. Reliable deployment, therefore, requires uncertainty signals that anticipate emerging failure dynamics during execution rather than reflect only instantaneous action entropy. We introduce \emph{GroundControl}, a trajectory-consistent uncertainty estimator defined as statistical deviation from nominal goal-directed distance-to-goal dynamics aggregated over an episode. GroundControl models distance evolution using a constant-velocity Kalman filter and combines normalized innovation statistics with complementary trajectory features capturing progress, monotonicity, path efficiency, and oscillatory behavior. The resulting uncertainty score reflects geometric and temporal inconsistency in navigation behavior rather than local prediction dispersion. To evaluate uncertainty quality independently of task success, we formalize \emph{Selective Risk--Coverage Navigation (SRCN)}, a protocol that measures how effectively an uncertainty score ranks episodes by failure or inefficiency using risk--coverage curves and AURC / E-AURC summaries. Across five EB-Navigation splits ($N=300$ episodes), trajectory-consistent uncertainty achieves near-oracle ordering under success-based selective risk, with weighted-average $\mathrm{E\text{-}AURC}_{\mathrm{SR}}=0.0024$ for the GPT-4o model, substantially outperforming entropy-, conformal-, and heuristic baselines. Under SPL-based selective evaluation, GroundControl consistently achieves the lowest AURC and E-AURC across models and navigation splits. These results show that modeling deviation from goal-directed dynamics provides an interpretable and robust signal for anticipating navigation failures in vision-language agents.

Problem

• vision-language navigation
• navigation failures
• trajectory-level breakdowns
• uncertainty estimation
• failure anticipation

Innovation

• trajectory-consistent uncertainty
• vision-language navigation
• Kalman filter
• risk-coverage evaluation
• failure anticipation