Beyond One-Way Influence: Bidirectional Opinion Dynamics in Multi-Turn Human-LLM Interactions

2025-10-22

AI Summary

Prior work predominantly models human–LLM interaction as unidirectional influence, neglecting dynamic bidirectional opinion evolution in multi-turn dialogue. Method: We conduct three controlled experiments—static statements, standard chatbots, and personalized chatbots—combined with fine-grained multi-turn dialogue analysis and quantitative tracking of stance shifts. Contribution/Results: We find human stances remain largely stable across interactions, whereas LLM outputs exhibit significant alignment toward user stances; personalization intensifies this bidirectional alignment, especially when users share personal narratives. This study is the first to systematically demonstrate that LLMs are not passive responders but actively adapt their outputs to user positions—a phenomenon we term “over-alignment.” Such adaptation introduces trade-offs between responsiveness and output stability, raising critical concerns for reliability in human–AI collaboration. Our findings provide theoretical grounding and design cautions for building trustworthy, adaptive conversational systems.

Technology Category

• Humans and AI: Intelligent User Interfaces
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Large language model (LLM)-powered chatbots are increasingly used for opinion exploration. Prior research examined how LLMs alter user views, yet little work extended beyond one-way influence to address how user input can affect LLM responses and how such bi-directional influence manifests throughout the multi-turn conversations. This study investigates this dynamic through 50 controversial-topic discussions with participants (N=266) across three conditions: static statements, standard chatbot, and personalized chatbot. Results show that human opinions barely shifted, while LLM outputs changed more substantially, narrowing the gap between human and LLM stance. Personalization amplified these shifts in both directions compared to the standard setting. Analysis of multi-turn conversations further revealed that exchanges involving participants' personal stories were most likely to trigger stance changes for both humans and LLMs. Our work highlights the risk of over-alignment in human-LLM interaction and the need for careful design of personalized chatbots to more thoughtfully and stably align with users.

Problem

• Investigating bidirectional opinion influence in human-LLM conversations
• Examining how personal stories trigger mutual stance changes
• Addressing over-alignment risks in personalized chatbot interactions

Innovation

• Bidirectional opinion dynamics in human-LLM interactions
• Personalized chatbot amplifies mutual stance changes
• Multi-turn conversations reveal story-triggered alignment shifts