Skinned Motion Retargeting with Dense Geometric Interaction Perception

2024-10-28

arXiv.org

AI Summary

To address geometric interaction artifacts—such as interpenetration, contact misalignment, and jitter—in skinned character motion retargeting, this paper proposes MeshRet, an end-to-end framework. Methodologically: (i) it introduces a Semantically Consistent Sensor (SCS) to establish dense mesh correspondences; (ii) it pioneers a Dynamic Mesh Interaction (DMI) field that unifies modeling of both contact and non-contact geometric relationships; and (iii) it designs a geometry-aware neural network that jointly optimizes contact preservation, self-interpenetration suppression, and motion semantic consistency, thereby abandoning conventional multi-stage correction pipelines. Evaluated on the Mixamo and ScanRet datasets, MeshRet achieves state-of-the-art performance, significantly eliminating geometric anomalies. The implementation is publicly available.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Computer Vision: Motion & Tracking
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Capturing and maintaining geometric interactions among different body parts is crucial for successful motion retargeting in skinned characters. Existing approaches often overlook body geometries or add a geometry correction stage after skeletal motion retargeting. This results in conflicts between skeleton interaction and geometry correction, leading to issues such as jittery, interpenetration, and contact mismatches. To address these challenges, we introduce a new retargeting framework, MeshRet, which directly models the dense geometric interactions in motion retargeting. Initially, we establish dense mesh correspondences between characters using semantically consistent sensors (SCS), effective across diverse mesh topologies. Subsequently, we develop a novel spatio-temporal representation called the dense mesh interaction (DMI) field. This field, a collection of interacting SCS feature vectors, skillfully captures both contact and non-contact interactions between body geometries. By aligning the DMI field during retargeting, MeshRet not only preserves motion semantics but also prevents self-interpenetration and ensures contact preservation. Extensive experiments on the public Mixamo dataset and our newly-collected ScanRet dataset demonstrate that MeshRet achieves state-of-the-art performance. Code available at https://github.com/abcyzj/MeshRet.

Problem

• Animation
• Motion Transition
• Body-part Interaction

Innovation

• MeshRet
• DMI field
• consistent body part interaction