PanoWorld: Real-World Panoramic Generation

2026-07-10

AI Summary

This work addresses the challenge of modeling long-range memory while preserving physical consistency in panoramic world models under large-scale spatial variations and complex lighting conditions. The authors propose a novel approach based on rotation-equivariant representations that simplifies camera trajectories to translational motion under a fixed orientation. By jointly modeling current actions and historical memory through Dense Panoramic Ray Conditioning (DPRC) and a Geometry-aware Memory Augmentation (GMA) mechanism, the method leverages a three-stage progressive training strategy to optimize the entire system. Key contributions include the first incorporation of rotation equivariance into panoramic world modeling to reduce trajectory complexity, the introduction of World360—the first large-scale hybrid real-simulated panoramic dataset—and state-of-the-art performance on this benchmark, demonstrating superior physical consistency and generation quality.

Abstract

In this work, we aim to address the challenge of long-range memory in panoramic world models by exploiting the rotation-equivariant property of omnidirectional representations, where rotation can be treated as an implicit geometric transformation.Building on this insight, we propose PanoWorld, which simplifies camera trajectories into translations via fixed headings for both current-action modeling and long-range memory through Dense Panoramic Ray-Conditioning (DPRC) and Geometry-aware Memory Augmentation (GMA).Then, a three-stage training pipeline is introduced to progressively optimize each component. To better evaluate physical consistency under large-scale spatial variations and diverse illumination conditions, where existing datasets are relatively stable, we construct World360, a large-scale dataset consisting of both real-world video clips collected via panoramic unmanned aerial vehicles and high-quality simulated clips generated by AirSim360.Extensive experiments on World360 demonstrate the effectiveness of PanoWorld, outperforming alternative methods by a large margin.Our models, training code, and dataset will be publicly available. More information can be found on our project page: https://lihaoy-ux.github.io/panoworld-page/.

Problem

• panoramic generation
• long-range memory
• rotation-equivariant
• physical consistency
• world models

Innovation

• rotation-equivariant
• panoramic generation
• long-range memory
• geometry-aware
• Dense Panoramic Ray-Conditioning