Hallucination in World Models is Predictable and Preventable

2026-06-25

AI Summary

This work addresses the hallucination problem commonly observed in generative world models during rollouts, which stems from dynamic bias due to insufficient coverage of the state-action space. For the first time, we attribute such hallucinations to inadequate state-action coverage and introduce a unified, lightweight signaling mechanism capable of simultaneously detecting and mitigating three canonical types of hallucination. Building upon this insight, we construct MMBench2, a large-scale benchmark dataset, and integrate coverage-aware sampling with curiosity-driven data collection to enable efficient model fine-tuning. Remarkably, with only 50 real-world trajectories, our approach enables rapid adaptation to novel environments and substantially improves generalization performance in unseen scenarios.

Abstract

Modern generative world models render increasingly realistic action-controllable futures, yet they frequently hallucinate: rollouts remain visually fluent while drifting from the ground-truth dynamics. We hypothesize that hallucination concentrates in low-coverage regions of the state-action space, where lightweight data-centric signals can both detect it and guide mitigation. To test this, we introduce MMBench2, a 427-hour, 210-task dataset for visual world modeling with ground-truth actions, rewards, and live simulators, and train a 350M-parameter world model on it. We identify three distinct hallucination modes: perceptual, action-marginalized, and scene-diverging -- each anchored to a different stage of the pipeline, and develop three signals that accurately predict where the model will fail. To close coverage gaps at training time, we develop a coverage-aware sampling technique; to close them online, our hallucination predictors serve as curiosity rewards for targeted data collection, yielding a data-efficient finetuning recipe that adapts the pretrained world model to entirely unseen environments with as few as 50 real environment trajectories. Overall, our findings reveal that hallucination in world models is inherently a data coverage issue, and that the same signals used to detect it can also be used for mitigation. An interactive web version of our paper is available at https://www.nicklashansen.com/mmbench2

Problem

• hallucination
• world models
• data coverage
• visual dynamics
• state-action space

Innovation

• world models
• hallucination detection
• data coverage
• curiosity-driven exploration
• MMBench2