Enhancing In-context Panoramic Generation via Geometric-aware Pretraining

2026-07-09

AI Summary

This work addresses the limitations of existing context-aware 360° panorama generation methods, which suffer from insufficient training data and poor geometric consistency and global coherence. To overcome these challenges, the authors propose Canvas360, a two-stage framework that first learns 3D structural priors through geometry-aware pretraining and then enables unified modeling of diverse tasks—including style transfer, inpainting, outpainting, and editing—via task-specific fine-tuning. Key contributions include the construction of Canvas360Dataset, a large-scale dataset comprising over one million high-quality paired panoramic images; a token-level task concatenation mechanism; and the novel integration of depth-parallel generation with similarity-based loss regularization to enhance geometric awareness. Experiments demonstrate that the proposed method significantly improves panoramic fidelity on metrics such as FAED and achieves state-of-the-art or competitive performance across multiple quantitative evaluations.

Abstract

In this work, we present Canvas360, a two-stage framework for in-context panoramic generation that combines geometry-aware pretraining with downstream task-specific fine-tuning. To address the lack of large-scale, high-quality training data tailored to in-context panoramic tasks, we propose Canvas360Dataset, a collection of 1M high-quality paired panoramic samples for style transfer, inpainting, outpainting, and editing, enabling effective supervision across diverse in-context generation scenarios. On the modeling side, Canvas360 enhances text-to-panorama generation through parallel depth generation, velocity circular padding, and similarity loss regularization, enabling the model to learn geometry-aware representations, capture object distortion details, and improve geometric consistency and global coherence. Furthermore, empowered by strong panoramic priors, Canvas360 enables a unified in-context panoramic generation framework that supports diverse downstream tasks via token-level concatenation, surpassing prior methods in both task coverage and modeling flexibility. Extensive experiments show that Canvas360 improves panoramic image fidelity, achieving particularly strong performance on the panorama-specific FAED metric and competitive or leading results across the reported quantitative evaluations. More information can be found on our project page: https://zry000.github.io/Canvas360/

Problem

• in-context panoramic generation
• geometric consistency
• panoramic image fidelity
• large-scale training data
• global coherence

Innovation

• geometry-aware pretraining
• in-context panoramic generation
• Canvas360Dataset
• parallel depth generation
• token-level concatenation