WiT: Waypoint Diffusion Transformers via Trajectory Conflict Navigation

2026-03-16

AI Summary

This work addresses trajectory conflicts and suboptimal generation arising from semantic discontinuities in pixel space, which entangle optimal transport paths. To resolve this, the authors propose the Waypoint Diffusion Transformer (WiT), which decouples the generation process into a two-segment path by introducing intermediate semantic waypoints guided by a pretrained vision model. This approach explicitly models trajectories in pixel space, circumventing information loss inherent in latent-space formulations. WiT incorporates a novel Just-Pixel AdaLN mechanism that enables continuous, waypoint-driven conditioning of the Transformer backbone. Built upon the Flow Matching and diffusion frameworks, WiT outperforms strong baselines on ImageNet at 256×256 resolution and accelerates JiT training convergence by a factor of 2.2.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Planning, Routing, and Scheduling: Planning with Language Models
• Search and Optimization: Learning to Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

While recent Flow Matching models avoid the reconstruction bottlenecks of latent autoencoders by operating directly in pixel space, the lack of semantic continuity in the pixel manifold severely intertwines optimal transport paths. This induces severe trajectory conflicts near intersections, yielding sub-optimal solutions. Rather than bypassing this issue via information-lossy latent representations, we directly untangle the pixel-space trajectories by proposing Waypoint Diffusion Transformers (WiT). WiT factorizes the continuous vector field via intermediate semantic waypoints projected from pre-trained vision models. It effectively disentangles the generation trajectories by breaking the optimal transport into prior-to-waypoint and waypoint-to-pixel segments. Specifically, during the iterative denoising process, a lightweight generator dynamically infers these intermediate waypoints from the current noisy state. They then continuously condition the primary diffusion transformer via the Just-Pixel AdaLN mechanism, steering the evolution towards the next state, ultimately yielding the final RGB pixels. Evaluated on ImageNet 256x256, WiT beats strong pixel-space baselines, accelerating JiT training convergence by 2.2x. Code will be publicly released at https://github.com/hainuo-wang/WiT.git.

Problem

• trajectory conflicts
• pixel-space generation
• optimal transport
• semantic continuity
• Flow Matching

Innovation

• Waypoint Diffusion Transformers
• Trajectory Disentanglement
• Pixel-space Diffusion
• Just-Pixel AdaLN
• Semantic Waypoints