Human-in-the-Loop Optimization with Model-Informed Priors

2025-10-08

AI Summary

To address the challenges of scarce prior knowledge, high iteration costs with real users, and poor scalability in human-computer interaction (HCI) optimization, this paper proposes HOMI—the first unified framework integrating in-situ optimization with model-guided simulation-based optimization. HOMI pretrains an optimizer on high-fidelity synthetic user data, eliminating reliance on costly real-user feedback. It introduces Neural Acquisition Function+ (NAF+), a novel acquisition mechanism that jointly leverages Bayesian optimization, neural networks, and reinforcement learning to learn adaptive querying policies end-to-end from synthetic data. Evaluated on a virtual reality mid-air keyboard optimization task, HOMI reduces real-user iterations by 62% and accelerates convergence by 2.3× compared to baseline methods. The framework significantly enhances the efficiency, generalizability, and practicality of human-in-the-loop optimization.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Humans and AI: Learning Human Values and Preferences
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Human-in-the-loop optimization identifies optimal interface designs by iteratively observing user performance. However, it often requires numerous iterations due to the lack of prior information. While recent approaches have accelerated this process by leveraging previous optimization data, collecting user data remains costly and often impractical. We present a conceptual framework, Human-in-the-Loop Optimization with Model-Informed Priors (HOMI), which augments human-in-the-loop optimization with a training phase where the optimizer learns adaptation strategies from diverse, synthetic user data generated with predictive models before deployment. To realize HOMI, we introduce Neural Acquisition Function+ (NAF+), a Bayesian optimization method featuring a neural acquisition function trained with reinforcement learning. NAF+ learns optimization strategies from large-scale synthetic data, improving efficiency in real-time optimization with users. We evaluate HOMI and NAF+ with mid-air keyboard optimization, a representative VR input task. Our work presents a new approach for more efficient interface adaptation by bridging in situ and in silico optimization processes.

Problem

• Accelerates human-in-the-loop optimization using model-informed priors
• Reduces costly user data collection through synthetic training data
• Improves real-time interface adaptation efficiency with neural acquisition functions

Innovation

• Uses model-informed priors for optimization
• Neural acquisition function trained with reinforcement learning
• Learns strategies from large-scale synthetic data