Walking the Schr""odinger Bridge: A Direct Trajectory for Text-to-3D Generation

2025-11-06

AI Summary

To address artifacts—such as oversaturation and over-smoothing—in text-to-3D generation caused by Score Distillation Sampling (SDS), this paper proposes TraCe, a novel framework grounded in Schrödinger Bridge (SB) theory. We reformulate SDS as the SB inverse process, establishing a differentiable and traceable optimal transport path from current rendered images to the target text-conditioned distribution. To realize this, we introduce Trajectory-Centered Distillation, which leverages LoRA to efficiently learn dynamic score fields along the SB trajectory, while employing low-classifier-free-guidance-scale (CFG) sampling for stable text conditioning. Experiments demonstrate that TraCe significantly improves geometric detail, texture fidelity, and generation stability across multiple benchmarks, achieving state-of-the-art performance with reduced sampling overhead.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Generation
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Recent advancements in optimization-based text-to-3D generation heavily rely on distilling knowledge from pre-trained text-to-image diffusion models using techniques like Score Distillation Sampling (SDS), which often introduce artifacts such as over-saturation and over-smoothing into the generated 3D assets. In this paper, we address this essential problem by formulating the generation process as learning an optimal, direct transport trajectory between the distribution of the current rendering and the desired target distribution, thereby enabling high-quality generation with smaller Classifier-free Guidance (CFG) values. At first, we theoretically establish SDS as a simplified instance of the Schr""odinger Bridge framework. We prove that SDS employs the reverse process of an Schr""odinger Bridge, which, under specific conditions (e.g., a Gaussian noise as one end), collapses to SDS's score function of the pre-trained diffusion model. Based upon this, we introduce Trajectory-Centric Distillation (TraCe), a novel text-to-3D generation framework, which reformulates the mathematically trackable framework of Schr""odinger Bridge to explicitly construct a diffusion bridge from the current rendering to its text-conditioned, denoised target, and trains a LoRA-adapted model on this trajectory's score dynamics for robust 3D optimization. Comprehensive experiments demonstrate that TraCe consistently achieves superior quality and fidelity to state-of-the-art techniques.

Problem

• Addresses artifacts in text-to-3D generation from diffusion models
• Formulates generation as optimal transport between rendering and target distributions
• Introduces trajectory-centric distillation for robust 3D optimization

Innovation

• Uses Schrödinger Bridge framework for trajectory optimization
• Introduces Trajectory-Centric Distillation for text-to-3D generation
• Trains LoRA-adapted model on trajectory score dynamics