Modeling and Optimizing User Preferences in AI Copilots: A Comprehensive Survey and Taxonomy

2025-05-28

AI Summary

Existing research lacks a systematic integration of how AI copilots detect, interpret, and adapt to users’ personalized preferences to enhance experience, trust, and productivity. Method: This paper introduces the first comprehensive preference optimization framework for AI copilots, structured around three phases—pre-interaction, in-interaction, and post-interaction—and establishes a unified taxonomy. It bridges personalized AI, human-AI collaboration, and large language model (LLM) adaptation, formally defining “AI copilot.” The methodology integrates implicit/explicit signal acquisition, intent modeling, feedback-driven closed-loop adaptation, LLM personalization via fine-tuning, and explainability analysis. Contribution/Results: We deliver a structured preference resource ontology and a method selection guide, providing both theoretical foundations and practical design principles for developing adaptive, trustworthy, and productivity-enhancing preference-aware AI collaborators.

Technology Category

• Humans and AI: Learning Human Values and Preferences
• Machine Learning: Learning Preferences or Rankings
• Knowledge Representation and Reasoning: Preferences

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search

Abstract

AI copilots, context-aware, AI-powered systems designed to assist users in tasks such as software development and content creation, are becoming integral to modern workflows. As these systems grow in capability and adoption, personalization has emerged as a cornerstone for ensuring usability, trust, and productivity. Central to this personalization is preference optimization: the ability of AI copilots to detect, interpret, and align with individual user preferences. While personalization techniques are well-established in domains like recommender systems and dialogue agents, their adaptation to interactive, real-time systems like AI copilots remains fragmented and underexplored. This survey addresses this gap by synthesizing research on how user preferences are captured, modeled, and refined within the design of AI copilots. We introduce a unified definition of AI copilots and propose a phase-based taxonomy of preference optimization strategies, structured around pre-interaction, mid-interaction, and post-interaction stages. We analyze techniques for acquiring preference signals, modeling user intent, and integrating feedback loops, highlighting both established approaches and recent innovations. By bridging insights from AI personalization, human-AI collaboration, and large language model adaptation, this survey provides a structured foundation for designing adaptive, preference-aware AI copilots. It offers a holistic view of the available preference resources, how they can be leveraged, and which technical approaches are most suited to each stage of system design.

Problem

• Optimizing AI copilots to align with individual user preferences
• Developing real-time personalization techniques for interactive AI systems
• Creating a taxonomy for preference optimization in AI copilot design

Innovation

• Phase-based taxonomy for preference optimization strategies
• Techniques for acquiring and modeling user preferences
• Integration of feedback loops in AI copilots