FACT: Failure-Aware Causal Training for World-Action Models

2026-08-10

AI Summary

This work addresses a critical limitation in existing world-action models, which predominantly rely on successful demonstration data and consequently struggle to accurately predict the outcomes of erroneous actions, often leading to future state hallucinations and biased action evaluations. To overcome this, the authors propose a causal world-action model that introduces, for the first time, a failure-aware training mechanism. This approach employs an action-conditioned interface to jointly predict future video frames and task progress, explicitly supervised by failure trajectories to model action consequences. Integrating action-conditioned video prediction, inverse dynamics modeling, and task progress estimation, the method significantly outperforms baseline approaches in both simulated and real-world bimanual manipulation tasks. Performance consistently improves with increased failure data, effectively mitigating success-biased hallucinations under erroneous actions and enabling accurate success-failure assessment of candidate actions during inference.

Abstract

Recent world-action models (WAMs) show that co-training policies with future prediction can provide physical priors for action generation. Building on the future-prediction ability of video models, many WAMs generate future videos and recover actions with inverse-dynamics models, or use these predicted videos as goal conditions for action generation. In both cases, the world model is trained mostly on successful demonstrations and has little reason to predict the consequences of bad actions. We introduce FACT, a causal World-Action Model that predicts future video and task progress conditioned on the executed action. This action-conditioned interface allows failure rollouts to supervise action consequences, turning bad actions into valid future targets rather than being discarded. Failure-aware training makes the progress predictor aware of both successful and failed action outcomes, which can optionally be used to score sampled action candidates at inference. Extensive experiments on simulation and real-world bimanual manipulation tasks show that FACT outperforms many existing baselines, improves as failure data are incorporated into training, and reduces success-biased future hallucination under bad actions. See more details at https://fact-wam.github.io/

Problem

• World-Action Models
• failure-aware learning
• action-conditioned prediction
• future hallucination
• bimanual manipulation

Innovation

• Failure-Aware Training
• Causal World-Action Model
• Action-Conditioned Prediction
• Inverse Dynamics
• Future Video Prediction