SegmentDreamer: Towards High-fidelity Text-to-3D Synthesis with Segmented Consistency Trajectory Distillation

2025-07-07

AI Summary

To address conditional guidance bias in consistency distillation (CD) for text-to-3D generation—arising from imbalance between self-consistency and cross-consistency—this paper proposes Segmented Consistency Trajectory Distillation (SCTD). SCTD partitions the probability flow ordinary differential equation (ODE) trajectory into multiple segments, enforcing consistency constraints independently within each segment; theoretically, this yields a tighter upper bound on distillation error. Integrated with score-based distillation sampling, consistency-distillation optimization, and 3D Gaussian Splatting (3DGS) rendering, SCTD establishes an efficient and stable end-to-end generation pipeline. Experiments demonstrate that our method surpasses current state-of-the-art approaches in visual quality, geometric accuracy, and detail fidelity. It significantly enhances controllability and realism in text-driven 3D content generation.

Technology Category

• Computer Vision: 3D Computer Vision
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Recent advancements in text-to-3D generation improve the visual quality of Score Distillation Sampling (SDS) and its variants by directly connecting Consistency Distillation (CD) to score distillation. However, due to the imbalance between self-consistency and cross-consistency, these CD-based methods inherently suffer from improper conditional guidance, leading to sub-optimal generation results. To address this issue, we present SegmentDreamer, a novel framework designed to fully unleash the potential of consistency models for high-fidelity text-to-3D generation. Specifically, we reformulate SDS through the proposed Segmented Consistency Trajectory Distillation (SCTD), effectively mitigating the imbalance issues by explicitly defining the relationship between self- and cross-consistency. Moreover, SCTD partitions the Probability Flow Ordinary Differential Equation (PF-ODE) trajectory into multiple sub-trajectories and ensures consistency within each segment, which can theoretically provide a significantly tighter upper bound on distillation error. Additionally, we propose a distillation pipeline for a more swift and stable generation. Extensive experiments demonstrate that our SegmentDreamer outperforms state-of-the-art methods in visual quality, enabling high-fidelity 3D asset creation through 3D Gaussian Splatting (3DGS).

Problem

• Address imbalance in self- and cross-consistency for 3D generation
• Improve fidelity in text-to-3D synthesis via segmented distillation
• Enhance visual quality and stability in 3D asset creation

Innovation

• Segmented Consistency Trajectory Distillation for balance
• Partitioned PF-ODE trajectory for tighter error bound
• Enhanced distillation pipeline for swift stable generation