Do Vision-Language-Action Models Mean What They Say? On the Role of Faithfulness in Embodied Reasoning

2026-07-06

AI Summary

This work addresses the limited faithfulness of reasoning chains in existing vision-language-action models—i.e., whether these chains genuinely reflect the agent’s decision-making process—a shortcoming that hinders generalization. The authors operationalize faithfulness as a learnable behavioral criterion and introduce the Pinocchio critic, a mechanism trained via reinforcement learning with dense reward signals to enhance grounding in environmental observations and coherence across reasoning steps. Through human evaluation, trajectory prediction, and counterfactual testing, the method demonstrates improved faithfulness by 4% and 18% over current alignment and trajectory-error-based post-training approaches, respectively, while maintaining downstream task performance. Notably, it also exhibits a 1.6× improvement in responsiveness under out-of-distribution counterfactual scenarios.

Abstract

Embodied Chain-of-Thought has emerged as a promising mechanism to enhance robot decision-making and interpretability in black-box Vision-Language Action (VLA) models. However, whether this verbalized Chain-of-Thought truthfully reflects the policy's underlying decision process remains poorly understood. We distinguish between functional reasoning, in which reasoning improves task performance, and faithful reasoning, in which reasoning truly reflects the policy's internal decision process. We argue that SoTA alignment strategies offer a necessary but insufficient notion of faithfulness, admitting reasoning whose intermediate steps can mask the causal links in action prediction through confounding factors (e.g., reasoning that is ungrounded in the environment and internally disconnected or inconsistent), restricting policy generalization. We study this gap through a human evaluation of a SoTA reasoning model for autonomous driving, revealing an inconsistent coupling between reasoning quality and downstream trajectory improvement. We then operationalize a behavioral surrogate for embodied faithfulness through a learned critic, Pinocchio, scoring observation grounding and stepwise coherence, and use this critic as a dense reward signal in post-training an embodied policy with reinforcement learning. Across withheld driving benchmarks, our post-trained planner improves faithfulness by 4% and 18% over SoTA alignment and trajectory error post-training baselines, respectively, while maintaining competitive downstream task performance. Finally, on a synthetic out-of-distribution test set, post-training for faithfulness improves policy responsiveness to rare counterfactual scenarios by 1.6x that of a SoTA policy, suggesting that faithful reasoning traces contribute to more robust, generalizable, and interpretable embodied intelligence. Project page: https://mjf-su.github.io/pinocchio/

Problem

• Vision-Language-Action Models
• Faithfulness
• Embodied Reasoning
• Chain-of-Thought
• Policy Generalization

Innovation

• embodied faithfulness
• Vision-Language-Action models
• Chain-of-Thought reasoning
• reinforcement learning post-training
• reasoning grounding