Motion Keyframe Interpolation for Any Human Skeleton via Temporally Consistent Point Cloud Sampling and Reconstruction

2024-05-13

European Conference on Computer Vision

AI Summary

Existing supervised keyframe interpolation methods rely on large-scale motion datasets with fixed skeletal topologies, severely limiting generalization to unseen skeletons and practical applicability. This paper proposes the first unsupervised cross-skeleton keyframe interpolation framework. It decouples motion representation from skeletal structure via temporally consistent point cloud sampling and reconstruction. To ensure motion coherence, we introduce a temporal K-nearest neighbors loss; to encode motion increments robustly, we design First-Frame Offset Quaternions (FOQ); and to enhance reconstruction stability, we incorporate Rest-Pose Augmentation (RPA). Crucially, our method requires no annotated motion data for target skeletons, enabling high-fidelity zero-shot interpolation on unseen skeletons. Evaluated on heterogeneous skeletal models—including SMPL and Mixamo—our approach significantly outperforms supervised baselines, achieving state-of-the-art performance in joint accuracy and motion naturalness.

Technology Category

• Computer Vision: Motion & Tracking
• Machine Learning: Unsupervised & Self-Supervised Learning
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

In the character animation field, modern supervised keyframe interpolation models have demonstrated exceptional performance in constructing natural human motions from sparse pose definitions. As supervised models, large motion datasets are necessary to facilitate the learning process; however, since motion is represented with fixed hierarchical skeletons, such datasets are incompatible for skeletons outside the datasets' native configurations. Consequently, the expected availability of a motion dataset for desired skeletons severely hinders the feasibility of learned interpolation in practice. To combat this limitation, we propose Point Cloud-based Motion Representation Learning (PC-MRL), an unsupervised approach to enabling cross-compatibility between skeletons for motion interpolation learning. PC-MRL consists of a skeleton obfuscation strategy using temporal point cloud sampling, and an unsupervised skeleton reconstruction method from point clouds. We devise a temporal point-wise K-nearest neighbors loss for unsupervised learning. Moreover, we propose First-frame Offset Quaternion (FOQ) and Rest Pose Augmentation (RPA) strategies to overcome necessary limitations of our unsupervised point cloud-to-skeletal motion process. Comprehensive experiments demonstrate the effectiveness of PC-MRL in motion interpolation for desired skeletons without supervision from native datasets.

Problem

• Enables motion interpolation for any human skeleton
• Eliminates need for skeleton-specific motion datasets
• Uses unsupervised point cloud-based learning approach

Innovation

• Unsupervised point cloud motion representation learning
• Temporal point cloud sampling for skeleton obfuscation
• First-frame Offset Quaternion for motion reconstruction