HINT-Plan: Human Intention-Aware Robot Task Planning in Context-Rich Environments using Vision Language Models

2026-09-15

AI Summary

本文提出HINT-Plan，利用视觉语言模型预测人类意图并结合场景图进行机器人任务规划，以解决复杂环境中人机协作任务规划问题。

Abstract

Approaches to incorporating human awareness into mobile robot decision-making mainly focus on collision avoidance in low-level motion planning, often overlooking the challenges posed by human presence and high-level behavior. To address this vacancy, we present HINT-Plan, a novel approach to integrate human intention prediction into robot task planning. HINT-Plan employs Vision Language Models (VLMs) to anticipate high-level human intentions from third-person image observations, convert them into goal states, and solve joint task-planning problems. To effectively enable scene awareness in context-rich environments, we use hierarchical Scene Graphs (SGs) as high-level representations of the environment, and translate environmental topology and actionable knowledge into formal planning language to ensure executable plans. Evaluated in a photorealistic simulation, HINT-Plan achieves an overall success rate of 69.71% in joint human-robot task planning, substantially outperforming the baselines by up to 35.29%, while also reducing functional conflicts. The results show the effectiveness of explicitly incorporating inferred human intentions into formal multi-agent task planning for proactive human-aware robot decision-making.

Problem

• Human Intention
• Robot Task Planning
• Context-Rich Environments

Innovation

• Human Intention Prediction
• Vision Language Models
• Hierarchical Scene Graphs
• Joint Task Planning