Self-attention vector output similarities reveal how machines pay attention

2025-12-26

AI Summary

This study investigates the functional specialization and dynamic evolution of self-attention heads in BERT-12. To this end, we propose a novel method for constructing context similarity matrices from attention head output vectors, integrating dot-product similarity metrics, intra- and inter-layer visualization, and statistical modeling to systematically quantify head-specific responses to lexical repetition, contextual co-occurrence, and sentence boundaries. Our analysis reveals four key findings: (1) inter-layer similarity patterns evolve from long-range to short-range dependencies across layers; (2) deeper-layer heads exhibit strong, selective activation at sentence separators; (3) attention heads functionally differentiate—specializing respectively in modeling lexical repetition, capturing high-frequency word contexts, or encoding local intra-sentence associations; and (4) we provide the first empirical evidence of token-centered focusing behavior. These results establish a quantifiable theoretical foundation and analytical framework for interpreting self-attention mechanisms in transformer-based language models.

Technology Category

• Natural Language Processing: Lexical Semantics and Morphology
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Machine Learning: Unsupervised & Self-Supervised Learning

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

The self-attention mechanism has significantly advanced the field of natural language processing, facilitating the development of advanced language-learning machines. Although its utility is widely acknowledged, the precise mechanisms of self-attention underlying its advanced learning and the quantitative characterization of this learning process remains an open research question. This study introduces a new approach for quantifying information processing within the self-attention mechanism. The analysis conducted on the BERT-12 architecture reveals that, in the final layers, the attention map focuses on sentence separator tokens, suggesting a practical approach to text segmentation based on semantic features. Based on the vector space emerging from the self-attention heads, a context similarity matrix, measuring the scalar product between two token vectors was derived, revealing distinct similarities between different token vector pairs within each head and layer. The findings demonstrated that different attention heads within an attention block focused on different linguistic characteristics, such as identifying token repetitions in a given text or recognizing a token of common appearance in the text and its surrounding context. This specialization is also reflected in the distribution of distances between token vectors with high similarity as the architecture progresses. The initial attention layers exhibit substantially long-range similarities; however, as the layers progress, a more short-range similarity develops, culminating in a preference for attention heads to create strong similarities within the same sentence. Finally, the behavior of individual heads was analyzed by examining the uniqueness of their most common tokens in their high similarity elements. Each head tends to focus on a unique token from the text and builds similarity pairs centered around it.

Problem

• Quantifying self-attention mechanisms in language models
• Analyzing attention heads' focus on linguistic features
• Characterizing token similarity evolution across model layers

Innovation

• Quantifying self-attention via vector output similarity matrices
• Analyzing attention head specialization for linguistic features
• Observing layer progression from long-range to sentence-focused similarities