TAPESTRY: From Geometry to Appearance via Consistent Turntable Videos

2026-03-18

AI Summary

This work addresses the challenge of automatically generating realistic and view-consistent appearances for untextured 3D models—a critical task in digital content creation. The authors propose a geometry-conditioned video diffusion approach that leverages multimodal geometric feature encoding to constrain the generation of 360-degree turntable videos. A novel 3D-aware inpainting mechanism is introduced to reconstruct self-occluded regions, ensuring complete surface coverage. By explicitly integrating geometric priors into the video diffusion framework, the method produces high-quality, temporally coherent dynamic previews suitable for direct use in UV texture back-projection or as supervision for neural rendering pipelines. Experimental results demonstrate significant improvements over existing techniques in both view consistency and 3D reconstruction fidelity, enabling fully automated production of ready-to-use 3D assets.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Automatically generating photorealistic and self-consistent appearances for untextured 3D models is a critical challenge in digital content creation. The advancement of large-scale video generation models offers a natural approach: directly synthesizing 360-degree turntable videos (TTVs), which can serve not only as high-quality dynamic previews but also as an intermediate representation to drive texture synthesis and neural rendering. However, existing general-purpose video diffusion models struggle to maintain strict geometric consistency and appearance stability across the full range of views, making their outputs ill-suited for high-quality 3D reconstruction. To this end, we introduce TAPESTRY, a framework for generating high-fidelity TTVs conditioned on explicit 3D geometry. We reframe the 3D appearance generation task as a geometry-conditioned video diffusion problem: given a 3D mesh, we first render and encode multi-modal geometric features to constrain the video generation process with pixel-level precision, thereby enabling the creation of high-quality and consistent TTVs. Building upon this, we also design a method for downstream reconstruction tasks from the TTV input, featuring a multi-stage pipeline with 3D-Aware Inpainting. By rotating the model and performing a context-aware secondary generation, this pipeline effectively completes self-occluded regions to achieve full surface coverage. The videos generated by TAPESTRY are not only high-quality dynamic previews but also serve as a reliable, 3D-aware intermediate representation that can be seamlessly back-projected into UV textures or used to supervise neural rendering methods like 3DGS. This enables the automated creation of production-ready, complete 3D assets from untextured meshes. Experimental results demonstrate that our method outperforms existing approaches in both video consistency and final reconstruction quality.

Problem

• 3D appearance generation
• geometric consistency
• turntable videos
• texture synthesis
• neural rendering

Innovation

• geometry-conditioned video diffusion
• turntable video generation
• 3D-aware inpainting
• consistent appearance synthesis
• neural rendering supervision