ChainHOI: Joint-based Kinematic Chain Modeling for Human-Object Interaction Generation

2025-03-17

AI Summary

Existing text-driven human-object interaction (HOI) generation methods rely on implicit whole-body pose modeling, leading to biomechanically implausible and semantically inaccurate interactions. Method: This paper proposes the first explicit joint-level and kinematic-chain-level modeling of HOI relationships. We construct a joint graph to encode geometric and semantic associations between human joints and objects, and introduce a Kinematics-Informed Interaction Module (KIM) that incorporates human skeletal dynamics priors. These components are integrated into an end-to-end trainable Generative Spatio-Temporal Graph Convolutional Network (GST-GCN). Contribution/Results: Our framework overcomes the limitations of conventional tokenized full-pose modeling. Evaluated on two public benchmarks, it achieves significant improvements over state-of-the-art methods in quantitative metrics (e.g., FID, PCKh, contact accuracy), while qualitative results demonstrate more realistic motion synthesis, higher semantic fidelity in object interaction, and enhanced biomechanical plausibility.

Technology Category

• Humans and AI: Interaction Techniques and Devices
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

We propose ChainHOI, a novel approach for text-driven human-object interaction (HOI) generation that explicitly models interactions at both the joint and kinetic chain levels. Unlike existing methods that implicitly model interactions using full-body poses as tokens, we argue that explicitly modeling joint-level interactions is more natural and effective for generating realistic HOIs, as it directly captures the geometric and semantic relationships between joints, rather than modeling interactions in the latent pose space. To this end, ChainHOI introduces a novel joint graph to capture potential interactions with objects, and a Generative Spatiotemporal Graph Convolution Network to explicitly model interactions at the joint level. Furthermore, we propose a Kinematics-based Interaction Module that explicitly models interactions at the kinetic chain level, ensuring more realistic and biomechanically coherent motions. Evaluations on two public datasets demonstrate that ChainHOI significantly outperforms previous methods, generating more realistic, and semantically consistent HOIs. Code is available href{https://github.com/qinghuannn/ChainHOI}{here}.

Problem

• Explicitly models joint-level human-object interactions
• Introduces joint graph for capturing object interactions
• Ensures biomechanically coherent motions via kinematic modeling

Innovation

• Joint graph for object interaction modeling
• Generative Spatiotemporal Graph Convolution Network
• Kinematics-based Interaction Module for realistic motions