Lagrangian Motion Fields for Long-term Motion Generation

2024-09-03

IEEE Transactions on Pattern Analysis and Machine Intelligence

AI Summary

Long-term motion generation suffers from temporal redundancy and insufficient dynamic modeling due to frame-based representations. To address this, we propose the Lagrangian Motion Field (LaMoF), the first approach to introduce physics-inspired Lagrangian particle modeling into motion generation: joints are modeled as short-horizon uniformly moving particles, enabling a compact, interpretable “hyper-motion” representation that unifies spatial structure and temporal dynamics. LaMoF eliminates neural preprocessing and frame dependency by parameterizing motion via geometric motion fields and encoding motion manifolds through non-learnable, analytically derived flows. This yields a lightweight, cross-modal paradigm supporting infinite looping, fine-grained controllability, and universal applicability. Evaluated on music-driven dance and text-to-motion generation, LaMoF significantly improves long-range coherence, visual realism, and motion diversity while reducing computational cost—enabling real-time, controllable synthesis and seamless looping.

Technology Category

• Computer Vision: Motion & Tracking
• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Long-term motion generation is a challenging task that requires producing coherent and realistic sequences over extended durations. Current methods primarily rely on framewise motion representations, which capture only static spatial details and overlook temporal dynamics. This approach leads to significant redundancy across the temporal dimension, complicating the generation of effective long-term motion. To overcome these limitations, we introduce the novel concept of Lagrangian Motion Fields, specifically designed for long-term motion generation. By treating each joint as a Lagrangian particle with uniform velocity over short intervals, our approach condenses motion representations into a series of "supermotions" (analogous to superpixels). This method seamlessly integrates static spatial information with interpretable temporal dynamics, transcending the limitations of existing network architectures and motion sequence content types. Our solution is versatile and lightweight, eliminating the need for neural network preprocessing. Our approach excels in tasks such as long-term music-to-dance generation and text-to-motion generation, offering enhanced efficiency, superior generation quality, and greater diversity compared to existing methods. Additionally, the adaptability of Lagrangian Motion Fields extends to applications like infinite motion looping and fine-grained controlled motion generation, highlighting its broad utility. Video demonstrations are available at https://plyfager.github.io/LaMoG.

Problem

• Generating coherent long-term motion sequences with temporal dynamics
• Overcoming redundancy in framewise motion representations for efficiency
• Creating versatile motion generation without neural network preprocessing

Innovation

• Lagrangian Motion Fields treat joints as particles
• Condenses motion into supermotions for efficiency
• Versatile lightweight approach without neural preprocessing