GIRAF: Towards Generalizable Human Interactions with Articulated Objects

2026-07-08

AI Summary

This work addresses the challenge of generating coherent, realistic, and generalizable full-body human motions that interact with articulated objects, encompassing navigation, fine-grained hand–object contact, and object articulation dynamics. To this end, it introduces the first unified text-conditioned diffusion model that jointly models full-body motion, hand–object contact, and articulated object dynamics. The approach leverages an object-centric representation, a hybrid-domain training strategy, and contact-aware data augmentation to enhance realism and generalization. Evaluated on unseen object geometries and configurations, the method demonstrates superior generalization capabilities and significantly outperforms current state-of-the-art approaches in generating physically plausible and semantically consistent interactions.

Abstract

Synthesizing realistic full-body human interactions with articulated objects is a fundamental challenge for embodied AI and graphics, with applications in robotics training and virtual agents. Existing models remain limited: some focus on simple activities with static objects, while others restrict attention to hand-only manipulation. This leaves open the problem of generating coordinated full-body motion that approaches, manipulates, and moves articulated objects in a realistic and generalizable way. The key difficulty lies in reasoning jointly about locomotion, fine-grained contact, and object articulation. Models must capture subtle hand-object correspondences that transfer across object geometries, while also producing seamless transitions from navigation to manipulation. At the same time, the scarcity of large-scale paired motion-scene data makes it difficult to generalize across diverse object positions and shapes. We introduce a text-conditioned diffusion model that addresses these challenges through three core ideas: an object-centric representation that unifies hand-object contact with object surfaces, a mixed-domain training strategy that balances locomotion and interaction, and a contact-based augmentation scheme that expands training diversity. Through experiments, our method demonstrated strong generalization to unseen object configurations, surpassing current state-of-the-art methods.

Problem

• human-object interaction
• articulated objects
• full-body motion
• generalization
• embodied AI

Innovation

• diffusion model
• object-centric representation
• contact-based augmentation
• full-body human-object interaction
• generalizable motion synthesis