CaliTex: Geometry-Calibrated Attention for View-Coherent 3D Texture Generation

2025-11-26

AI Summary

Existing diffusion-based 3D texture generation methods suffer from cross-view inconsistency, primarily due to the geometry-agnostic nature of full attention mechanisms—causing geometric ambiguity and unstable appearance-structure coupling. To address this, we propose a geometry-calibrated attention mechanism: Part-Aligned Attention enforces part-level structural alignment, while Condition-Routed Attention enables geometry-conditioned, cross-modal information routing. Our method introduces a two-stage diffusion Transformer architecture that explicitly incorporates geometric priors into attention computation, thereby embedding geometric consistency as an intrinsic network property. This yields seamless multi-view rendering and spatially coherent textures across diverse scenes. Quantitatively, our approach achieves significant improvements in cross-view consistency over both public and commercial baselines: +23.6% reduction in FID and +18.4% increase in CLIP-Score.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Despite major advances brought by diffusion-based models, current 3D texture generation systems remain hindered by cross-view inconsistency -- textures that appear convincing from one viewpoint often fail to align across others. We find that this issue arises from attention ambiguity, where unstructured full attention is applied indiscriminately across tokens and modalities, causing geometric confusion and unstable appearance-structure coupling. To address this, we introduce CaliTex, a framework of geometry-calibrated attention that explicitly aligns attention with 3D structure. It introduces two modules: Part-Aligned Attention that enforces spatial alignment across semantically matched parts, and Condition-Routed Attention which routes appearance information through geometry-conditioned pathways to maintain spatial fidelity. Coupled with a two-stage diffusion transformer, CaliTex makes geometric coherence an inherent behavior of the network rather than a byproduct of optimization. Empirically, CaliTex produces seamless and view-consistent textures and outperforms both open-source and commercial baselines.

Problem

• Addresses cross-view inconsistency in 3D texture generation systems
• Solves attention ambiguity causing geometric confusion in textures
• Aligns attention mechanisms with 3D structure for spatial fidelity

Innovation

• Geometry-calibrated attention aligns with 3D structure
• Part-Aligned Attention enforces spatial cross-view alignment
• Condition-Routed Attention routes appearance via geometry pathways