Learning User Preferences Through Interaction for Long-Term Collaboration

2026-01-06

arXiv.org

AI Summary

This work addresses the critical challenge of enabling conversational agents to continuously learn and adapt to user preferences over long-term, multi-turn interactions to enhance collaboration quality and user experience. We propose MultiSessionCollab—the first benchmark specifically designed for user preference learning in extended multi-session collaborative settings—and introduce an agent architecture equipped with an evolvable persistent memory mechanism. This agent leverages implicit behavioral feedback generated by a user simulator to drive memory updates and reflective reasoning. Experimental results demonstrate that our approach significantly improves task success rates and interaction efficiency while reducing user burden. Furthermore, a human-subject study validates its effectiveness in real-world scenarios, confirming measurable improvements in user experience.

Technology Category

• Humans and AI: Learning Human Values and Preferences
• Multiagent Systems: Multiagent Learning
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• User Modeling, Personalization and Recommendation: Studies of user behavior, including longitudinal effects of personalized systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search

Abstract

As conversational agents accumulate experience collaborating with users, adapting to user preferences is essential for fostering long-term relationships and improving collaboration quality over time. We introduce MultiSessionCollab, a benchmark that evaluates how well agents can learn user preferences and leverage them to improve collaboration quality throughout multiple sessions. To develop agents that succeed in this setting, we present long-term collaborative agents equipped with a memory that persists and refines user preference as interaction experience accumulates. Moreover, we demonstrate that learning signals can be derived from user simulator behavior in MultiSessionCollab to train agents to generate more comprehensive reflections and update their memory more effectively. Extensive experiments show that equipping agents with memory improves long-term collaboration, yielding higher task success rates, more efficient interactions, and reduced user effort. Finally, we conduct a human user study that demonstrates that memory helps improve user experience in real-world settings.

Problem

• user preferences
• long-term collaboration
• conversational agents
• interactive learning
• personalization

Innovation

• long-term collaboration
• user preference learning
• persistent memory
• multi-session interaction
• conversational agents