Ink3D: Sculpting 3D Assets with Extremely Complex Textures via Video Generative Models

2026-07-01

AI Summary

Existing 3D generation methods struggle to reproduce complex textures from a single image, primarily due to the scarcity of large-scale 3D training data with high-fidelity appearance. This work proposes a geometry–texture disentangled generation framework: it first leverages an off-the-shelf 3D generative model to obtain a coarse geometry (i.e., a white-matter mesh), then synthesizes a multi-view appearance video using a newly introduced conditional video generation model, OrbitPainter. A neural texture baking module, TextureOptimizer, subsequently fuses multi-view information to enhance texture consistency. Notably, this is the first approach to integrate large-scale pre-trained video generation models into 3D texture synthesis, achieving significantly improved detail fidelity and cross-view consistency without requiring any 3D texture supervision, thereby outperforming current state-of-the-art methods.

Abstract

Recent 3D generative models can synthesize high-quality geometry but often struggle to reproduce intricate textures from reference images, largely due to the scarcity of large-scale 3D training data with rich surface appearance. In contrast, visual generative models are trained on datasets several orders of magnitude larger and excel at modeling complex visual patterns. Motivated by this gap, we introduce Ink3D, a framework that bridges 3D generation with large-scale video generative models to synthesize extremely complex textures. Ink3D first reconstructs a white-mesh geometry using an off-the-shelf 3D generation model. It then employs OrbitPainter, a conditional video generative model, to produce dense orbit-scan videos capturing object appearance across viewpoints. To convert these views into coherent textures, we introduce TextureOptimizer, a neural baking module that integrates dense multi-view observations while mitigating geometry inconsistencies arising from video generation. By decoupling geometry and texture synthesis and leveraging large-scale pretrained video priors, Ink3D enables significantly richer and more faithful texture generation than prior approaches.

Problem

• 3D texture generation
• complex textures
• 3D generative models
• surface appearance
• texture synthesis

Innovation

• video generative models
• 3D texture synthesis
• OrbitPainter
• TextureOptimizer
• neural baking