Your Vision-Language-Action Model Already Has Attention Heads For Path Deviation Detection

2026-03-14

AI Summary

This work addresses the susceptibility of Vision-Language-Action (VLA) models to trajectory deviations caused by visual reasoning hallucinations during navigation. While existing approaches rely on additional training or complex heuristics, this study reveals for the first time that VLA models inherently contain a “navigation head” with spatiotemporal causal awareness. By monitoring just three frozen attention heads within this module, the authors construct a training-free, real-time deviation detection framework. Upon detecting anomalies, the system seamlessly switches to a lightweight reinforcement learning policy for recovery. Evaluated on a physical robot, the method achieves a 44.6% true positive detection rate with only an 11.7% false positive rate, demonstrating its efficiency, practicality, and robustness.

Technology Category

• Computer Vision: Language and Vision
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Vision-Language-Action (VLA) models have demonstrated strong potential for predicting semantic actions in navigation tasks, demonstrating the ability to reason over complex linguistic instructions and visual contexts. However, they are fundamentally hindered by visual-reasoning hallucinations that lead to trajectory deviations. Addressing this issue has conventionally required training external critic modules or relying on complex uncertainty heuristics. In this work, we discover that monitoring a few attention heads within a frozen VLA model can accurately detect path deviations without incurring additional computational overhead. We refer to these heads, which inherently capture the spatiotemporal causality between historical visual sequences and linguistic instructions, as Navigation Heads. Using these heads, we propose an intuitive, training-free anomaly-detection framework that monitors their signals to detect hallucinations in real time. Surprisingly, among over a thousand attention heads, a combination of just three is sufficient to achieve a 44.6 % deviation detection rate with a low false-positive rate of 11.7 %. Furthermore, upon detecting a deviation, we bypass the heavy VLA model and trigger a lightweight Reinforcement Learning (RL) policy to safely execute a shortest-path rollback. By integrating this entire detection-to-recovery pipeline onto a physical robot, we demonstrate its practical robustness. All source code will be publicly available.

Problem

• Vision-Language-Action models
• path deviation
• visual-reasoning hallucinations
• trajectory deviation
• navigation

Innovation

• Vision-Language-Action models
• attention heads
• path deviation detection
• hallucination monitoring
• training-free anomaly detection