Preference Heads in Large Language Models: A Mechanistic Framework for Interpretable Personalization

2026-04-24

AI Summary

While large language models possess implicit personalization capabilities, their underlying mechanisms remain opaque, and existing approaches rely heavily on prompt engineering or fine-tuning. This work proposes DPS, a training-free, inference-time personalization framework that identifies sparse attention heads encoding user preferences through causal masking analysis. By contrasting model outputs with and without these “preference heads,” DPS enhances personalization in a controllable and efficient manner. The method offers the first interpretability-driven insight into how personalization is realized within Transformer architectures, integrating causal intervention, Preference Contribution Scoring (PCS), and contrastive logic. Experiments across multiple mainstream models demonstrate that DPS significantly improves personalization fidelity while preserving textual coherence. The code has been publicly released.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Natural Language Processing: (Large) Language Models
• Knowledge Representation and Reasoning: Preferences

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Large Language Models (LLMs) exhibit strong implicit personalization ability, yet most existing approaches treat this behavior as a black box, relying on prompt engineering or fine tuning on user data. In this work, we adopt a mechanistic interpretability perspective and hypothesize the existence of a sparse set of Preference Heads, attention heads that encode user specific stylistic and topical preferences and exert a causal influence on generation. We introduce Differential Preference Steering (DPS), a training free framework that (1) identifies Preference Heads through causal masking analysis and (2) leverages them for controllable and interpretable personalization at inference time. DPS computes a Preference Contribution Score (PCS) for each attention head, directly measuring its causal impact on user aligned outputs. During decoding, we contrast model predictions with and without Preference Heads, amplifying the difference between personalized and generic logits to selectively strengthen preference aligned continuations. Experiments on widely used personalization benchmarks across multiple LLMs demonstrate consistent gains in personalization fidelity while preserving content coherence and low computational overhead. Beyond empirical improvements, DPS provides a mechanistic explanation of where and how personalization emerges within transformer architectures. Our implementation is publicly available.

Problem

• Preference Heads
• Large Language Models
• Personalization
• Mechanistic Interpretability
• Attention Heads

Innovation

• Preference Heads
• Differential Preference Steering
• Mechanistic Interpretability
• Causal Masking Analysis
• Personalization