ProVoice: Designing Proactive Functionality for In-Vehicle Conversational Assistants using Multi-Objective Bayesian Optimization to Enhance Driver Experience

2026-01-27

AI Summary

This study addresses the challenge of personalizing proactive intervention strategies for in-vehicle conversational assistants to balance drivers’ mental workload, system predictability, and utility. The authors propose a human-in-the-loop multi-objective Bayesian optimization (HITL-MOBO) approach that automatically explores and refines intervention policies within a virtual reality driving simulator. By integrating human feedback into the optimization loop, this work pioneers the application of multi-objective Bayesian optimization to proactive in-vehicle dialogue systems. Through Pareto front analysis, the method effectively identifies optimal trade-offs that simultaneously minimize mental workload, enhance predictability, and maintain practical utility, enabling efficient and personalized discovery of intervention strategies tailored to individual drivers.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search

Abstract

The next step for In-vehicle Conversational Assistants (IVCAs) will be their capability to initiate and automate proactive system interactions throughout journeys. However, diverse drivers make it challenging to design voice interventions tailored towards individual on-road expectations. This paper evaluates the effectiveness of Human-in-the-Loop (HITL) Multi-Objective Bayesian Optimization (MOBO) in design by implementing ProVoice: a Virtual Reality (VR) driving simulator integrating MOBO to investigate the effects of IVCA design variants on perceived mental demand, predictability, and usefulness. By reporting the Pareto Front from a within-subjects VR study (N=19), this paper proposes optimal design trade-offs. Follow-up analysis demonstrates MOBO's success in discovering effective intervention strategies, with reduced participant mental demand, alongside enhanced predictability and usefulness while engaging with the proactive IVCA. Implications for computational techniques in future research on proactive intervention strategies are discussed. ProVoice can extend to include alternative design parameters and driving scenarios, encouraging intervention design on a broad scale.

Problem

• In-vehicle Conversational Assistants
• Proactive Interaction
• Driver Experience
• Personalization
• Mental Demand

Innovation

• Multi-Objective Bayesian Optimization
• Human-in-the-Loop
• Proactive Conversational Assistant
• Virtual Reality Driving Simulator
• Pareto Front