CubeDiff: Repurposing Diffusion-Based Image Models for Panorama Generation

2025-01-28

AI Summary

This work addresses the problem of generating high-quality 360° panoramic images from text or input images. We propose a cubemap-based multi-view diffusion model that represents the panorama as six orthogonal faces, jointly synthesizing all faces to avoid distortions inherent in spherical projections and bottlenecks of autoregressive modeling. To our knowledge, this is the first end-to-end adaptation of standard multi-view diffusion models to cubemap panoramas—achieved without explicit inter-face attention mechanisms. The approach enables fine-grained textual control and exhibits strong generalization capability. Quantitatively and qualitatively, our method achieves state-of-the-art performance, significantly outperforming equirectangular projection–based and autoregressive baselines in geometric consistency, resolution fidelity, and visual quality.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

We introduce a novel method for generating 360{deg} panoramas from text prompts or images. Our approach leverages recent advances in 3D generation by employing multi-view diffusion models to jointly synthesize the six faces of a cubemap. Unlike previous methods that rely on processing equirectangular projections or autoregressive generation, our method treats each face as a standard perspective image, simplifying the generation process and enabling the use of existing multi-view diffusion models. We demonstrate that these models can be adapted to produce high-quality cubemaps without requiring correspondence-aware attention layers. Our model allows for fine-grained text control, generates high resolution panorama images and generalizes well beyond its training set, whilst achieving state-of-the-art results, both qualitatively and quantitatively. Project page: https://cubediff.github.io/

Problem

• 360-degree panorama
• image synthesis
• text-to-image conversion

Innovation

• CubeDiff
• Diffusion Model
• 360-degree Panoramic Photos