HOID-R1: Reinforcement Learning for Open-World Human-Object Interaction Detection Reasoning with Multimodal Large Language Model

2025-08-15

AI Summary

Existing open-vocabulary human-object interaction (HOI) detection methods over-rely on large language models’ (LLMs) textual prompting capabilities while neglecting their untapped 3D spatial reasoning potential, thereby limiting interaction understanding accuracy in AR/VR and robotic applications. To address this, we propose the first multimodal reinforcement learning framework that jointly integrates chain-of-thought-guided supervised fine-tuning (SFT) and group-relative policy optimization (GRPO). Our approach innovatively introduces a multi-reward signal mechanism and a “LLM-as-judge” strategy to effectively suppress hallucination, enhance cross-modal alignment, and improve generalization. The framework performs end-to-end joint optimization of visual, linguistic, and spatial representations. It achieves state-of-the-art performance on standard benchmarks including HICO-DET and V-COCO, and demonstrates significant gains in zero-shot transfer to novel, open-world scenarios.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Understanding and recognizing human-object interaction (HOI) is a pivotal application in AR/VR and robotics. Recent open-vocabulary HOI detection approaches depend exclusively on large language models for richer textual prompts, neglecting their inherent 3D spatial understanding capabilities. To address this shortcoming, we introduce HOID-R1, the first HOI detection framework that integrates chain-of-thought (CoT) guided supervised fine-tuning (SFT) with group relative policy optimization (GRPO) within a reinforcement learning (RL) paradigm. Specifically, we initially apply SFT to imbue the model with essential reasoning capabilities, forcing the model to articulate its thought process in the output. Subsequently, we integrate GRPO to leverage multi-reward signals for policy optimization, thereby enhancing alignment across diverse modalities. To mitigate hallucinations in the CoT reasoning, we introduce an "MLLM-as-a-judge" mechanism that supervises the CoT outputs, further improving generalization. Extensive experiments show that HOID-R1 achieves state-of-the-art performance on HOI detection benchmarks and outperforms existing methods in open-world generalization to novel scenarios.

Problem

• Enhances 3D spatial understanding in HOI detection
• Integrates reinforcement learning for multimodal alignment
• Reduces hallucinations in chain-of-thought reasoning

Innovation

• Integrates CoT-guided SFT with GRPO in RL
• Uses MLLM-as-a-judge to supervise CoT outputs
• Enhances alignment across diverse modalities