ToMiE: Towards Modular Growth in Enhanced SMPL Skeleton for 3D Human with Animatable Garments

2024-10-10

arXiv.org

AI Summary

The SMPL model struggles to represent complex 3D geometries—such as handheld objects and loose clothing—that exhibit motion decoupling from the underlying body. Method: We propose an adaptive, modular skeletal growth framework that enables dynamic expansion of the kinematic tree. Our approach introduces a novel modular bone mechanism supporting joint-tree growth, jointly optimizing external joint poses and Linear Blend Skinning (LBS) weights in SE(3) space. It further incorporates motion-kernel-guided gradient localization, differentiable rendering, and explicit animation control. Contribution/Results: Unlike conventional unified parametric models, our framework significantly improves expressivity and controllability for decoupled components. Experiments demonstrate superior rendering fidelity and more robust animation generation across diverse challenging apparel scenarios—including voluminous garments and articulated accessories. The framework establishes a general, extensible paradigm for enhanced SMPL-based modeling, enabling scalable integration of anatomically and physically plausible non-rigid elements.

Technology Category

• Computer Vision: Motion & Tracking
• Search and Optimization: Sampling/Simulation-based Search
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

In this paper, we highlight a critical yet often overlooked factor in most 3D human tasks, namely modeling humans with complex garments. It is known that the parameterized formulation of SMPL is able to fit human skin; while complex garments, e.g., hand-held objects and loose-fitting garments, are difficult to get modeled within the unified framework, since their movements are usually decoupled with the human body. To enhance the capability of SMPL skeleton in response to this situation, we propose a modular growth strategy that enables the joint tree of the skeleton to expand adaptively. Specifically, our method, called ToMiE, consists of parent joints localization and external joints optimization. For parent joints localization, we employ a gradient-based approach guided by both LBS blending weights and motion kernels. Once the external joints are obtained, we proceed to optimize their transformations in SE(3) across different frames, enabling rendering and explicit animation. ToMiE manages to outperform other methods across various cases with garments, not only in rendering quality but also by offering free animation of grown joints, thereby enhancing the expressive ability of SMPL skeleton for a broader range of applications.

Problem

• Modeling 3D humans with hand-held objects
• Reconstructing avatars with loose-fitting clothing
• Enhancing SMPL skeleton for decoupled movements

Innovation

• Growth strategy for SMPL skeleton expansion
• Gradient-based parent joints localization
• SE(3) optimization for external joints