Bridging Physical Reasoning and Task Generalization via Visual Action Outcome Reasoning Alignment

2026-07-07

AI Summary

Existing vision-language models struggle to generalize to unseen tasks and environments in interactive physical reasoning, often generating hallucinated predictions that violate physical laws and exhibit misalignment between reasoning and actions. To address this, this work proposes VAORA, a novel approach featuring a dual-path alignment reward mechanism—comprising visual alignment and vision-action alignment rewards—that anchors the reasoning process to the current visual context and its resulting actions, thereby effectively suppressing hallucinations and bridging the gap between reasoning and behavior. By leveraging a pretrained expert agent to estimate task success probability, VAORA provides stable and dense reinforcement learning rewards. Experiments demonstrate that VAORA significantly improves generalization performance on the PHYRE and VirtualTool benchmarks, validating its effectiveness in advancing generalizable physical intelligence.

Abstract

Vision-language models (VLMs) struggle to generalize in interactive physical reasoning, particularly under unseen tasks and environments. Two key failure modes are prominent: hallucinated chain-of-thought (CoT) reasoning that contradicts physical reality, and misalignment between the model's reasoning and actions. We present VAORA (Visual Action Outcome Reasoning Alignment), a novel reward design that directly addresses both issues. VAORA introduces two complementary rewards: Visual Alignment Reward, which anchors VLM reasoning to the visual context independent of the agent action itself, and Visual-Action Alignment Reward, which grounds reasoning in the visual outcome induced by the model's action. Together, these rewards suppress hallucinated CoT and reduce the gap between reasoning and behavior. To improve training stability, we further employ smooth, dense rewards by estimating success probabilities using a pre-trained in-domain expert agent. Experiments on PHYRE and Virtual Tool support our performances across novel-task and unseen-environment settings, confirming that grounded and generalizable physical intelligence can be induced through VAORA.

Problem

• physical reasoning
• task generalization
• vision-language models
• hallucinated reasoning
• reasoning-action misalignment

Innovation

• Visual Action Outcome Reasoning Alignment
• physical reasoning
• vision-language models
• reward design
• task generalization