Who Responds When the Driver Is Gone? A Framework for Human Intent Understanding

2026-07-06

AI Summary

This work addresses the limited capability of existing autonomous driving systems to perceive and respond to passengers’ implicit intentions. We propose Intent2Drive, a framework that models passenger intent as a latent cognitive state jointly shaped by linguistic cues, personality traits, emotional states, behavioral patterns, and contextual factors. To support this approach, we introduce the Holistic Intent Dataset—the first benchmark integrating both explicit and implicit intent signals. Grounded in theory of mind, our architecture features a human intention reasoner and a hierarchical intention-conditioned planner, enabling end-to-end alignment from intent understanding to driving behavior generation. Experimental results demonstrate that our method significantly improves intent inference and goal construction while maintaining strong closed-loop planning performance, thereby advancing autonomous driving toward truly passenger-responsive systems.

Abstract

As autonomous vehicles progress toward fully driverless mobility, a critical question emerges: who understands and responds to passengers when the human driver is absent? Existing autonomous driving systems primarily optimize predefined navigation and control objectives from external scene observations, but they remain limited in perceiving and reasoning about in-cabin human intent. In this paper, we propose Intent2Drive, a unified framework for holistic human intent understanding and human-aligned planning. Instead of treating passenger intent as explicit commands alone, Intent2Drive models intent as a latent cognitive state shaped by language, personal attributes, emotional and physical conditions, behavioral signals, and situational context. To support this formulation, we construct a Holistic Intent Dataset (HID) that provides structured supervision over both explicit and implicit intent cues. Built upon HID, our Theory-of-Mind-inspired Human Intent Reasoner (HIR) infers a Latent Human State (LHS) and further translates it into a planner-compatible Human Intent Objective (HIO). We then introduce a Hierarchical Intent-Conditioned Planner (HICP) that incorporates HIO into route-level and trajectory-level planning, enabling driving behaviors to remain aligned with passenger needs across different planning horizons. Extensive experiments show that Intent2Drive improves structured human intent inference and HIO construction while preserving competitive closed-loop planning performance. These results demonstrate a promising step toward passenger-responsive autonomous driving systems that can reason about, interpret, and act upon human intent in driverless mobility.

Problem

• human intent understanding
• autonomous driving
• passenger-responsive systems
• driverless mobility
• in-cabin perception

Innovation

• human intent understanding
• latent cognitive state
• Theory-of-Mind
• intent-conditioned planning
• driverless mobility