Transition Models: Rethinking the Generative Learning Objective

2025-09-04

AI Summary

Generative modeling faces a fundamental trade-off between sample quality and computational efficiency: iterative diffusion models achieve high fidelity at prohibitive computational cost, whereas few-step models are inherently constrained by an upper bound on achievable quality. This paper introduces Transition Models (TiM), the first framework to formulate generative dynamics via an analytically tractable finite-time state transition equation, modeling synthesis as an exact continuous-time dynamical system. TiM unifies single-step generation and multi-step refinement, guaranteeing strictly monotonic quality improvement with increasing sampling steps—thereby breaking the quality ceiling of conventional few-step approaches. Employing native high-resolution training, TiM achieves state-of-the-art performance across all step counts, surpassing SD3.5 (8B parameters) and FLUX.1 (12B parameters) despite using only 0.865B parameters, and delivers exceptional fidelity at 4096×4096 resolution.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

A fundamental dilemma in generative modeling persists: iterative diffusion models achieve outstanding fidelity, but at a significant computational cost, while efficient few-step alternatives are constrained by a hard quality ceiling. This conflict between generation steps and output quality arises from restrictive training objectives that focus exclusively on either infinitesimal dynamics (PF-ODEs) or direct endpoint prediction. We address this challenge by introducing an exact, continuous-time dynamics equation that analytically defines state transitions across any finite time interval. This leads to a novel generative paradigm, Transition Models (TiM), which adapt to arbitrary-step transitions, seamlessly traversing the generative trajectory from single leaps to fine-grained refinement with more steps. Despite having only 865M parameters, TiM achieves state-of-the-art performance, surpassing leading models such as SD3.5 (8B parameters) and FLUX.1 (12B parameters) across all evaluated step counts. Importantly, unlike previous few-step generators, TiM demonstrates monotonic quality improvement as the sampling budget increases. Additionally, when employing our native-resolution strategy, TiM delivers exceptional fidelity at resolutions up to 4096x4096.

Problem

• Resolving trade-off between diffusion model fidelity and computational cost
• Overcoming restrictive training objectives limiting few-step generation quality
• Enabling continuous-time transitions for adaptive multi-step generative trajectories

Innovation

• Exact continuous-time dynamics equation for transitions
• Adapts to arbitrary-step transitions for generation
• Native-resolution strategy for high-fidelity outputs