An Irreducible Quantum Advantage in Aligning World Models with Reality

2026-08-20

AI Summary

研究解决了复杂环境中经典世界模型无法准确对齐现实的问题，提出使用单个量子三能级系统（qutrit）构建量子世界模型以精确匹配真实世界的奖励估计和最优策略。

Abstract

World models provide digital simulacra of the true world, allowing agents to be trained and tested before costly real-world deployment. At each time step, they receive an action and generate an observation and reward matching the statistics of the true world. In complex environments where present outcomes depend on events far in the past, this requires memory. One might expect that, by increasing memory, we can always build a model accurately enough to align the optimal agent policies of the real and virtual worlds. We show that this is false for classical world models, even when the true world itself is classical. We construct true worlds for which every finite classical model fails along the same possible trajectory: it either loses the ability to distinguish actions when the true world clearly prefers one, or repeatedly assigns the highest expected reward to suboptimal actions. Its expected-reward estimates also retain a nonvanishing average error. In contrast, each such true world admits a quantum world model using a single qutrit that reproduces it exactly: its reward estimates and preferred actions always match those of the true world, ensuring that the optimal policies of the real and virtual worlds remain perfectly aligned.

Problem

• world models
• memory
• optimal policies
• classical models
• quantum advantage

Innovation

• quantum world model
• qutrit
• optimal policy alignment