Tiled Diffusion

2024-12-19

arXiv.org

AI Summary

Traditional image stitching methods suffer from poor scalability, and existing automated approaches are largely confined to single-image texture generation, failing to support cross-domain, multi-image collaborative seamless tiling. To address this, we propose Tiled Diffusion—the first diffusion-based framework natively extended to general-purpose image tiling generation. It supports arbitrary topological connections (e.g., self-tiling, many-to-many stitching) and diverse application domains including textures, 360° panoramas, and game assets. Methodologically, we introduce latent-space tiling constraints, inter-tile feature consistency regularization, and a multi-scale boundary fusion mechanism to ensure both visual seamlessness and semantic coherence. Experiments demonstrate substantial improvements in automation and creative flexibility for texture synthesis, image extrapolation, and 360° content generation, achieving high-quality, spatially consistent outputs across multiple tiling tasks.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Game Design — Procedural Content Generation & Storytelling

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Image tiling -- the seamless connection of disparate images to create a coherent visual field -- is crucial for applications such as texture creation, video game asset development, and digital art. Traditionally, tiles have been constructed manually, a method that poses significant limitations in scalability and flexibility. Recent research has attempted to automate this process using generative models. However, current approaches primarily focus on tiling textures and manipulating models for single-image generation, without inherently supporting the creation of multiple interconnected tiles across diverse domains. This paper presents Tiled Diffusion, a novel approach that extends the capabilities of diffusion models to accommodate the generation of cohesive tiling patterns across various domains of image synthesis that require tiling. Our method supports a wide range of tiling scenarios, from self-tiling to complex many-to-many connections, enabling seamless integration of multiple images. Tiled Diffusion automates the tiling process, eliminating the need for manual intervention and enhancing creative possibilities in various applications, such as seamlessly tiling of existing images, tiled texture creation, and 360{deg} synthesis.

Problem

• Automates image tiling across diverse domains.
• Enables seamless integration of multiple images.
• Supports complex tiling scenarios without manual intervention.

Innovation

• Extends diffusion models for cohesive tiling patterns
• Automates tiling process across diverse image domains
• Supports self-tiling to complex many-to-many connections