MOCHI: Motion Enhancement of Collaborative Human-object Interactions

2026-06-16

AI Summary

This work addresses the challenges posed by multi-source noise—such as hand-object contact misalignment, motion jitter, temporal inconsistency, and missing finger details—in multi-human-object interaction (MHOI) data. The authors propose MOCHI, a two-stage framework that first optimizes noisy poses to generate physically plausible and semantically consistent hand grasps, then employs a diffusion-based denoising model to refine full-body motion by integrating single-person motion priors with human-object and human-human interaction constraints. MOCHI is the first approach to jointly leverage physical plausibility, semantic consistency, and diffusion denoising, achieving significant improvements in reconstruction quality on both real-world and synthetic MHOI datasets. It generalizes across varying numbers of participants and interaction types, supports keyframe-driven generation and object geometry-aware refinement, and demonstrates strong robustness and practical applicability.

Abstract

Collaborative human-object interaction shows dynamic and complex movements that require mutual anticipation and continuous adjustment between participants and the shared object. Modeling such collaborative multi-human object interaction (MHOI) scenarios requires high-quality data acquisition as a foundational step; however, this is challenging due to the inherent complexity of MHOI where human-human and human-object interactions occur simultaneously. Such complexity leads to noisy MHOI captures characterized by several artifacts: contact misalignment between hands and objects, motion jitter and temporal inconsistencies in the captured sequences, and missing or incomplete finger-level articulation details. To address these challenges, we present MOCHI (MOtion Enhancement of Collaborative Human-object Interactions), a two-stage framework for enhancing noisy MHOI data. Our approach first generates physically plausible hand grasps through optimization from noisy body input, producing grasps that are both physically plausible and semantically consistent with the body pose, where these optimized grasps are extended into complete hand-object interaction sequences. Consequently, the full-body motion for all participants are refined through a diffusion-based noise optimization framework that uses single-person motion priors. During the optimization process, we introduce optimization objectives to encode human-object and human-human interaction information within these single-person priors. Experimental results demonstrate the effectiveness of our pipeline across diverse MHOI data, either acquired by existing capture methods or synthesized by generative models. We further show robustness of our system across varying numbers of participants and types of interactions, and demonstrate various applications including keyframe-based MHOI creation and data augmentation through varying object geometries.

Problem

• collaborative human-object interaction
• motion capture noise
• hand-object contact misalignment
• temporal inconsistency
• finger articulation

Innovation

• motion enhancement
• collaborative human-object interaction
• diffusion-based optimization
• physical plausibility
• motion priors