MILD: Mediator Agent System with Bidirectional Perception and Multi-Layered Alignment for Human-Vehicle Collaboration

2026-05-02

AI Summary

This study addresses the challenges of insufficient mutual awareness of intentions and states between human drivers and vehicles in partial automation, which often leads to coordination failures and excessive cognitive load. To this end, the authors propose MILD, a mediated intelligent agent system that enables bidirectional human-vehicle alignment through integrated in-cabin and external perception coupled with lightweight policy generation. The core innovations include redefining the human role from passive supervisor to active manager, introducing a multi-level alignment mechanism, and incorporating an auditable decision-making framework composed of a perception agent, a lightweight policy agent, an Evidence and Constraint-weighted Policy Optimization (ECPO) module, and a retrieval-augmented generation component. Experimental results on three public datasets demonstrate that MILD significantly outperforms baseline methods in perception accuracy, policy quality, and human-rated adequacy, comfort, and explainability.

Abstract

Prior studies report that partial driving automation can increase the cognitive demands on human drivers. This effect largely arises from human drivers' lack of transparent insight into the vehicle's intentions and decision logic, as well as from automated systems' limited awareness of the driver's dynamic state and preferences. This bidirectional misalignment undermines shared situational awareness and exacerbates coordination failures in human-vehicle interaction. To address these limitations, we argue for a paradigm shift that elevates the human role from passive supervisor to active manager. We introduce the Mediator-in-the-Loop-Driving (MILD) system, based on an agentic system architecture to facilitate synergistic human-vehicle collaboration. MILD integrates a perception agent for joint in-cabin and out-of-cabin understanding with a lightweight strategy agent that generates compliant and explainable action suggestions. To ensure these strategies are strictly aligned with safety regulations and human values, we develop Evidence- and Constraint-weighted Policy Optimization (ECPO). ECPO leverages automatic validators to steer the agent toward behaviors that are not only accurate but also structurally complete, substantiated by evidence, and free from constraint violations. Furthermore, a retrieval-augmented generation module dynamically incorporates constraints from traffic regulations, speed recommendations, and driver preferences into the decision loop. Field experiments across three open datasets demonstrate that MILD consistently outperforms baselines in both perception accuracy and strategy quality under auditable offline metrics, and yields higher human-rated policy adequacy, comfort, and explanation than baselines. This work offers a practical pathway for building auditable and aligned agents for human-vehicle collaborative driving.

Problem

• human-vehicle collaboration
• bidirectional misalignment
• shared situational awareness
• driver state awareness
• intention transparency

Innovation

• MILD
• bidirectional perception
• multi-layered alignment
• ECPO
• retrieval-augmented generation