STaR: Seamless Spatial-Temporal Aware Motion Retargeting with Penetration and Consistency Constraints

2025-04-09

AI Summary

Motion retargeting faces dual challenges of geometric interpenetration and temporal jitter, making it difficult to simultaneously ensure semantic fidelity, interpenetration-free articulation, and trajectory coherence. This paper proposes a spatio-temporal dual-module sequence-to-sequence framework: a spatial module incorporates dense shape encoding and limb-interpenetration constraint loss to enforce geometric plausibility; a temporal module employs a temporal Transformer augmented with multi-level trajectory smoothing constraints to enhance motion continuity. We introduce the first joint optimization mechanism for interpenetration avoidance and temporal consistency, enabling simultaneous geometric non-penetration and temporal stability within a single end-to-end inference pass. Evaluated on Mixamo and ScanRet datasets, our method significantly reduces interpenetration rates and consistently outperforms state-of-the-art approaches in motion naturalness, semantic fidelity, and temporal stability.

Technology Category

• Computer Vision: Motion & Tracking
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Intelligent Robots: Motion and Path Planning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Motion retargeting seeks to faithfully replicate the spatio-temporal motion characteristics of a source character onto a target character with a different body shape. Apart from motion semantics preservation, ensuring geometric plausibility and maintaining temporal consistency are also crucial for effective motion retargeting. However, many existing methods prioritize either geometric plausibility or temporal consistency. Neglecting geometric plausibility results in interpenetration while neglecting temporal consistency leads to motion jitter. In this paper, we propose a novel sequence-to-sequence model for seamless Spatial-Temporal aware motion Retargeting (STaR), with penetration and consistency constraints. STaR consists of two modules: (1) a spatial module that incorporates dense shape representation and a novel limb penetration constraint to ensure geometric plausibility while preserving motion semantics, and (2) a temporal module that utilizes a temporal transformer and a novel temporal consistency constraint to predict the entire motion sequence at once while enforcing multi-level trajectory smoothness. The seamless combination of the two modules helps us achieve a good balance between the semantic, geometric, and temporal targets. Extensive experiments on the Mixamo and ScanRet datasets demonstrate that our method produces plausible and coherent motions while significantly reducing interpenetration rates compared with other approaches.

Problem

• Ensures geometric plausibility in motion retargeting to prevent interpenetration.
• Maintains temporal consistency to avoid motion jitter during retargeting.
• Balances semantic, geometric, and temporal targets for seamless motion transfer.

Innovation

• Spatial module with dense shape representation
• Temporal module using transformer for consistency
• Combines penetration and consistency constraints