Using High-Level Patterns to Estimate How Humans Predict a Robot will Behave

2024-09-20

arXiv.org

AI Summary

Human predictions of robot behavior are inherently high-level and coarse-grained—focusing on semantic intent rather than precise trajectories—yet most existing interactive models assume accurate trajectory-level prediction, compromising safety and naturalness. To address this, we propose a behavior-level prediction framework grounded in second-order theory of mind: it maps human–robot trajectories into a discrete behavioral-type latent space and decodes them into semantically consistent vector fields, enabling robots to infer how humans predict their high-level intentions (e.g., “maintain lane”). This work is the first to systematically incorporate the cognitive characteristics of human predictive reasoning into human–robot interaction modeling. Experiments on both simulated and real-world driving datasets demonstrate that our framework achieves significantly higher fidelity to actual human judgments compared to conventional trajectory-prediction baselines, thereby enhancing interaction safety and fluency.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Human-Robot Interaction
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Humans interacting with robots often form predictions of what the robot will do next. For instance, based on the recent behavior of an autonomous car, a nearby human driver might predict that the car is going to remain in the same lane. It is important for the robot to understand the human's prediction for safe and seamless interaction: e.g., if the autonomous car knows the human thinks it is not merging -- but the autonomous car actually intends to merge -- then the car can adjust its behavior to prevent an accident. Prior works typically assume that humans make precise predictions of robot behavior. However, recent research on human-human prediction suggests the opposite: humans tend to approximate other agents by predicting their high-level behaviors. We apply this finding to develop a second-order theory of mind approach that enables robots to estimate how humans predict they will behave. To extract these high-level predictions directly from data, we embed the recent human and robot trajectories into a discrete latent space. Each element of this latent space captures a different type of behavior (e.g., merging in front of the human, remaining in the same lane) and decodes into a vector field across the state space that is consistent with the underlying behavior type. We hypothesize that our resulting high-level and course predictions of robot behavior will correspond to actual human predictions. We provide initial evidence in support of this hypothesis through proof-of-concept simulations, testing our method's predictions against those of real users, and experiments on a real-world interactive driving dataset.

Problem

• Estimate human predictions of robot behavior using high-level patterns.
• Develop a second-order theory of mind for robot-human interaction.
• Validate high-level behavior predictions through simulations and real-world data.

Innovation

• Second-order theory of mind approach
• Discrete latent space embedding
• High-level behavior prediction decoding