Trajectory Forcing: Structure-First Generation with Controllable Semantic Trajectories

2026-06-21

AI Summary

Existing generative models typically optimize only the final output, lacking semantic controllability over intermediate generation steps. This work proposes Trajectory Forcing, a novel framework that explicitly models the generation path as a semantically editable trajectory progressing from global layout to fine-grained details. The approach constructs a coarse-to-fine teacher hierarchy based on DINOv2 clustering and trains a hierarchical, step-conditioned flow-matching model to enable structured and editable intermediate state generation. Experiments demonstrate that the framework maintains high-fidelity image synthesis while supporting cross-scale local editing. Furthermore, it introduces trajectory-aware metrics for consistency and controllability, moving beyond conventional endpoint-only evaluations such as FID and thereby addressing their inherent limitations.

Abstract

Diffusion and flow-based generative models produce strong images, yet their controllability remains largely endpoint-centric: users specify conditions and receive final outputs, while the intermediate generative dynamics remain hidden. Recent methods have begun to exploit generation order and process decomposition to improve sample quality, but still treat intermediate states as internal computation rather than objects for interaction. We propose Trajectory Forcing (TF), a trajectory-centric framework that makes the generation path explicit, semantic, and editable. TF organizes synthesis as a sequence of semantically structured stages, progressing from global layout to object-, part-, and detail-level representations. Each stage produces a decodable latent state that can be inspected, evaluated, and locally edited before the next stage begins. To instantiate this path, we derive coarse-to-fine teacher hierarchies by clustering pretrained visual representations such as DINOv2, and train a hierarchy-conditioned one-step flow-matching model at each level. We further introduce trajectory-aware metrics that measure structural consistency and local controllability beyond endpoint quality metrics such as FID. Experiments show that TF achieves competitive sample quality while exposing coherent intermediate states and supporting localized edits across semantic levels. By shifting the focus from final images to the generative path itself, TF opens a route toward controllable, trajectory-aware image synthesis.

Problem

• controllable generation
• semantic trajectory
• intermediate dynamics
• trajectory-aware synthesis
• generative path

Innovation

• Trajectory Forcing
• semantic trajectory
• hierarchical generation
• flow-matching
• controllable synthesis