Less is More: Improving Motion Diffusion Models with Sparse Keyframes

2025-03-17

AI Summary

Existing motion diffusion models rely on dense frame sequences, leading to redundant computation and inefficient training, thereby limiting performance in text-driven motion generation. To address this, we propose a novel sparse keyframe-driven diffusion framework. Our approach introduces a dynamic keyframe masking mechanism, integrated with geometry-aware sparse modeling, kinematics-guided interpolation, dynamic mask refinement, and generative prior transfer—shifting the paradigm from dense-frame regression to keyframe-driven synthesis. The method substantially reduces computational overhead while maintaining high fidelity within ≤50 diffusion steps. It achieves state-of-the-art performance in text-motion alignment and motion realism, and demonstrates strong generalization across downstream tasks. This work establishes a new paradigm for efficient and controllable generative motion modeling.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Intelligent Robots: Motion and Path Planning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Recent advances in motion diffusion models have led to remarkable progress in diverse motion generation tasks, including text-to-motion synthesis. However, existing approaches represent motions as dense frame sequences, requiring the model to process redundant or less informative frames. The processing of dense animation frames imposes significant training complexity, especially when learning intricate distributions of large motion datasets even with modern neural architectures. This severely limits the performance of generative motion models for downstream tasks. Inspired by professional animators who mainly focus on sparse keyframes, we propose a novel diffusion framework explicitly designed around sparse and geometrically meaningful keyframes. Our method reduces computation by masking non-keyframes and efficiently interpolating missing frames. We dynamically refine the keyframe mask during inference to prioritize informative frames in later diffusion steps. Extensive experiments show that our approach consistently outperforms state-of-the-art methods in text alignment and motion realism, while also effectively maintaining high performance at significantly fewer diffusion steps. We further validate the robustness of our framework by using it as a generative prior and adapting it to different downstream tasks. Source code and pre-trained models will be released upon acceptance.

Problem

• Reduces training complexity by focusing on sparse keyframes.
• Improves motion realism and text alignment in generation tasks.
• Enhances performance with fewer diffusion steps and computational cost.

Innovation

• Sparse keyframes reduce redundant frame processing.
• Dynamic keyframe masking enhances informative frame prioritization.
• Efficient interpolation maintains high performance with fewer steps.