Multi-Turn Neural Transparency: Surfacing Neural Activations Improves User Calibration to LLM Behavioral Drift

2026-05-14

AI Summary

This study addresses the risk of behavioral drift—such as sycophancy or toxicity—in large language models during multi-turn dialogues, which users often fail to detect, leading to potential misinformation. The authors propose a neural transparency interface that, for the first time, integrates high-fidelity personality-behavior vectors (R² ≥ 0.9) from mechanistic interpretability with dynamic, multi-turn visualizations—including sunburst charts and drift panels—to render real-time changes in internal model activations. In a randomized controlled experiment with 246 participants, this approach significantly improved users’ accuracy in anticipating and evaluating model behavior (effect sizes d = −0.34 to −0.49) and outperformed static single-turn visualizations (d = −0.32), effectively mitigating overconfidence.

Abstract

Chatbot behavior is often opaque to users, as responses can shift unpredictably across a conversation, drifting toward sycophancy, toxicity, or other unsafe responses. This can leave users vulnerable, either being misled by overly agreeable AI or manipulated by a harmful chatbot that no longer behaves as intended. To address this, we introduce multi-turn neural transparency, an interface that surfaces an LLM's internal neural activations in real time to help users anticipate and recognize how behaviors change across turns. We construct behavioral vectors for six personality traits using methods from mechanistic interpretability, identifying directions in activation space that correlate with trait expression ($R^2 \geq 0.9$) via contrastive system prompts, and visualize trait expression using a sunburst and drift panel that updates at each turn. In a randomized controlled study (N = 246), participants predicted trait expression from a system prompt alone, then rated observed behavior after interacting with the chatbot for both assistant and role-play personas. We find that participants without visualization struggled to accurately evaluate traits (RMSE $\approx$ 0.6-0.7), while the inclusion of neural transparency significantly improved both anticipation and evaluation compared to no visualization (d = -0.34 to -0.49). The multi-turn dynamic visualization additionally outperformed the static single-turn visualization on holistic evaluation of model behavior (d = -0.32). Transparency also reduced overconfidence: participants without visualization grew more confident despite no gain in accuracy. These findings suggest that surfacing internal model representations to everyday users is a meaningful step toward more transparent and informed human-AI interaction.

Problem

• behavioral drift
• neural transparency
• large language models
• user calibration
• multi-turn dialogue

Innovation

• multi-turn neural transparency
• mechanistic interpretability
• behavioral drift
• neural activation visualization
• human-AI interaction