Generative Panoramic Image Stitching

2025-07-08

AI Summary

Traditional panoramic stitching struggles with severe ghosting and artifacts when aligning reference images exhibiting significant disparities in disparity, illumination, and style; meanwhile, existing generative models fail to ensure structural coherence and content fidelity across large-scale panoramas. To address this, we propose a diffusion-based inpainting model fine-tuned on multiple reference images, enabling high-fidelity, seamless panorama generation from a single input image via cross-image feature alignment and layout preservation. Our method innovatively integrates multi-source conditional guidance with fine-grained inpainting control. Evaluated on real-world datasets, it substantially outperforms state-of-the-art baselines: generated panoramas achieve breakthrough improvements in structural continuity, visual naturalness, and global content consistency. This work establishes a novel paradigm for generative panoramic stitching in complex, heterogeneous imaging scenarios.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

We introduce the task of generative panoramic image stitching, which aims to synthesize seamless panoramas that are faithful to the content of multiple reference images containing parallax effects and strong variations in lighting, camera capture settings, or style. In this challenging setting, traditional image stitching pipelines fail, producing outputs with ghosting and other artifacts. While recent generative models are capable of outpainting content consistent with multiple reference images, they fail when tasked with synthesizing large, coherent regions of a panorama. To address these limitations, we propose a method that fine-tunes a diffusion-based inpainting model to preserve a scene's content and layout based on multiple reference images. Once fine-tuned, the model outpaints a full panorama from a single reference image, producing a seamless and visually coherent result that faithfully integrates content from all reference images. Our approach significantly outperforms baselines for this task in terms of image quality and the consistency of image structure and scene layout when evaluated on captured datasets.

Problem

• Synthesize seamless panoramas from multiple varied reference images
• Overcome ghosting and artifacts in traditional stitching methods
• Generate large coherent regions using diffusion-based inpainting

Innovation

• Fine-tunes diffusion-based inpainting model
• Outpaints panorama from single reference image
• Preserves scene content and layout