Guiding Human-Object Interactions with Rich Geometry and Relations

2025-03-26

AI Summary

Existing HOI synthesis methods often simplify object representations using centroids or nearest points, neglecting geometric details and leading to physically implausible interactions and inaccurate relational modeling. To address this, we propose ROG, a diffusion-based framework tailored for high-fidelity interactive scenarios such as VR. First, we introduce boundary-focused keypoint sampling—coupled with an Interaction Distance Field (IDF)—to explicitly encode spatial constraints between humans and objects. Second, we design a spatiotemporal joint attention mechanism that jointly grounds action generation in relational semantics and geometric consistency. Evaluated on standard HOI synthesis benchmarks, ROG significantly outperforms state-of-the-art methods, achieving substantial improvements in physical plausibility, geometric fidelity, and semantic accuracy of synthesized human–object interactions.

Technology Category

• Humans and AI: Interaction Techniques and Devices
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Intelligent Robots: Human-Robot Interaction

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Human-object interaction (HOI) synthesis is crucial for creating immersive and realistic experiences for applications such as virtual reality. Existing methods often rely on simplified object representations, such as the object's centroid or the nearest point to a human, to achieve physically plausible motions. However, these approaches may overlook geometric complexity, resulting in suboptimal interaction fidelity. To address this limitation, we introduce ROG, a novel diffusion-based framework that models the spatiotemporal relationships inherent in HOIs with rich geometric detail. For efficient object representation, we select boundary-focused and fine-detail key points from the object mesh, ensuring a comprehensive depiction of the object's geometry. This representation is used to construct an interactive distance field (IDF), capturing the robust HOI dynamics. Furthermore, we develop a diffusion-based relation model that integrates spatial and temporal attention mechanisms, enabling a better understanding of intricate HOI relationships. This relation model refines the generated motion's IDF, guiding the motion generation process to produce relation-aware and semantically aligned movements. Experimental evaluations demonstrate that ROG significantly outperforms state-of-the-art methods in the realism and semantic accuracy of synthesized HOIs.

Problem

• Enhancing human-object interaction realism with detailed geometry
• Overcoming simplified object representation limitations in HOI synthesis
• Improving motion generation with relation-aware spatial-temporal modeling

Innovation

• Boundary-focused key points for object representation
• Interactive distance field capturing HOI dynamics
• Diffusion-based relation model with attention mechanisms