HHI-Assist: A Dataset and Benchmark of Human-Human Interaction in Physical Assistance Scenario

2025-09-12

AI Summary

Physical human–robot interaction poses a fundamental challenge in accurately predicting human motion due to environmental variability and complex coupled dynamics. To address this, we introduce HHI-Assist—the first dual-human cooperative motion capture dataset explicitly designed for physical assistance scenarios. We further propose a conditional Transformer-based diffusion model that explicitly incorporates coupled-dynamics priors; its attention mechanism captures inter-individual action dependencies, enabling robust generalization to unseen interaction contexts. Extensive experiments demonstrate that our model achieves significant improvements over state-of-the-art baselines in multi-step pose prediction. All components—including the dataset, model architecture, and implementation code—are fully open-sourced. This work establishes a foundational resource and methodological framework for learning assistive robot policies and modeling dynamic human–robot collaboration.

Technology Category

• Intelligent Robots: Human-Robot Interaction
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Computer Vision: Diffusion Models for Vision

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The increasing labor shortage and aging population underline the need for assistive robots to support human care recipients. To enable safe and responsive assistance, robots require accurate human motion prediction in physical interaction scenarios. However, this remains a challenging task due to the variability of assistive settings and the complexity of coupled dynamics in physical interactions. In this work, we address these challenges through two key contributions: (1) HHI-Assist, a dataset comprising motion capture clips of human-human interactions in assistive tasks; and (2) a conditional Transformer-based denoising diffusion model for predicting the poses of interacting agents. Our model effectively captures the coupled dynamics between caregivers and care receivers, demonstrating improvements over baselines and strong generalization to unseen scenarios. By advancing interaction-aware motion prediction and introducing a new dataset, our work has the potential to significantly enhance robotic assistance policies. The dataset and code are available at: https://sites.google.com/view/hhi-assist/home

Problem

• Predicting human motion in physical assistance interactions
• Addressing variability and complexity in assistive settings
• Improving robotic assistance through interaction-aware motion prediction

Innovation

• Dataset of human-human assistive interactions
• Transformer-based diffusion model for pose prediction
• Captures coupled dynamics in physical assistance