Retrieving and Refining Winning Noise Tickets for Diffusion-Based Motion Generation

2026-07-07

AI Summary

Existing diffusion-based text-to-motion generation models often suffer from semantic drift in long-horizon and compositional motions, struggling to balance semantic fidelity with motion coherence. This work proposes WINRO, a novel framework that reveals—for the first time—the decisive role of initial noise in determining the semantics of generated motions. WINRO introduces a training-free, model-agnostic mechanism for noise retrieval and refinement: by retrieving the most text-aligned “winning noise ticket” and applying KL-regularized optimization—optionally enhanced with a single-step forward LoRA adapter—it substantially improves text-motion alignment. Experiments demonstrate that WINRO effectively enhances semantic consistency for both MDM and MotionLCM on HumanML3D, boosts temporal robustness on the MTT benchmark, and generalizes successfully to motion stylization and spatially constrained motion synthesis tasks.

Abstract

Diffusion-based text-to-motion models synthesize realistic human motions but often exhibit semantic drift from the input text. Motion is inherently temporal, especially in compositional and long-duration sequences that require semantic consistency across multiple action segments and smooth kinematic transitions throughout the trajectory. We posit that the initial noise is central to this consistency: within the Gaussian noise space, certain instances, i.e. winning noise tickets, carry latent structure that biases denoising toward particular motion semantics, even under null prompts. We propose WInning Noise Retrieval and Optimization (WINRO), a training-free, model-agnostic framework that improves text-motion alignment by selecting and refining such tickets before diffusion sampling. WINRO maps random noises to motion features generated under null prompts, retrieves the best-aligned noise for a given text, and refines it via a KL-regularized objective that reduces the residual semantic gap while preserving the Gaussian prior. An optional LoRA-based adapter amortizes this refinement into a single forward pass. WINRO consistently improves text-motion fidelity across different base models, MDM and MotionLCM, on HumanML3D without retraining, improves temporal robustness on the MTT benchmark, and generalizes to applications such as motion stylization and spatial constraint satisfaction.

Problem

• semantic drift
• text-to-motion alignment
• temporal consistency
• diffusion-based motion generation
• motion semantics

Innovation

• winning noise tickets
• diffusion-based motion generation
• text-motion alignment
• training-free optimization
• KL-regularized refinement