How Do LLMs Persuade? Linear Probes Can Uncover Persuasion Dynamics in Multi-Turn Conversations

2025-08-07

AI Summary

This work investigates how large language models (LLMs) achieve persuasion in multi-turn dialogues, focusing on the dynamic process underlying persuasive success. Method: We propose the first linear probing framework for modeling multi-turn persuasion, grounded in cognitive science–inspired representational analysis. Our approach precisely localizes the temporal onset of persuasion success at both sample and dataset levels, and disentangles the interplay among target personality traits, strategy selection, and persuasive outcomes. Contribution/Results: Compared to conventional prompt engineering, our method achieves comparable or superior performance in identifying persuasive strategies while offering substantially higher computational efficiency—enabling scalable analysis of large dialogue corpora. By yielding interpretable, representation-level insights, this work establishes a novel, extensible paradigm for studying high-level social behaviors—such as persuasion, manipulation, and deception—driven by LLMs.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Large Language Models (LLMs) have started to demonstrate the ability to persuade humans, yet our understanding of how this dynamic transpires is limited. Recent work has used linear probes, lightweight tools for analyzing model representations, to study various LLM skills such as the ability to model user sentiment and political perspective. Motivated by this, we apply probes to study persuasion dynamics in natural, multi-turn conversations. We leverage insights from cognitive science to train probes on distinct aspects of persuasion: persuasion success, persuadee personality, and persuasion strategy. Despite their simplicity, we show that they capture various aspects of persuasion at both the sample and dataset levels. For instance, probes can identify the point in a conversation where the persuadee was persuaded or where persuasive success generally occurs across the entire dataset. We also show that in addition to being faster than expensive prompting-based approaches, probes can do just as well and even outperform prompting in some settings, such as when uncovering persuasion strategy. This suggests probes as a plausible avenue for studying other complex behaviours such as deception and manipulation, especially in multi-turn settings and large-scale dataset analysis where prompting-based methods would be computationally inefficient.

Problem

• Understanding how LLMs persuade in multi-turn conversations
• Using linear probes to analyze persuasion dynamics effectively
• Comparing probes with prompting methods for persuasion strategies

Innovation

• Linear probes analyze LLM persuasion dynamics
• Probes detect persuasion success and strategies
• Probes outperform prompting in some settings