Entropy-Lens: The Information Signature of Transformer Computations

2025-02-23

AI Summary

The opaque internal computation mechanisms of Transformers hinder their interpretability and robustness evaluation. To address this, we propose the first Shannon entropy-based framework for dynamic residual flow analysis, modeling inter-layer information entropy evolution as a universal “computational signature.” Our method requires no architectural priors and is applicable across multimodal Transformer architectures—including LLMs and ViTs—under black-box conditions. It achieves three key functionalities: (1) precise discrimination among mainstream model families; (2) automatic classification of task prompt types; and (3) accurate prediction of output accuracy—demonstrating strong correlation (ρ > 0.82) between entropy trajectories and accuracy across multiple benchmarks. Crucially, this work establishes information entropy evolution as a generalizable and transferable computational representation paradigm for Transformers—a novel contribution to foundational model analysis.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Computer Vision: Large Vision Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Transformer models have revolutionized fields from natural language processing to computer vision, yet their internal computational dynamics remain poorly understood raising concerns about predictability and robustness. In this work, we introduce Entropy-Lens, a scalable, model-agnostic framework that leverages information theory to interpret frozen, off-the-shelf large-scale transformers. By quantifying the evolution of Shannon entropy within intermediate residual streams, our approach extracts computational signatures that distinguish model families, categorize task-specific prompts, and correlate with output accuracy. We further demonstrate the generality of our method by extending the analysis to vision transformers. Our results suggest that entropy-based metrics can serve as a principled tool for unveiling the inner workings of modern transformer architectures.

Problem

• Understand Transformer internal computational dynamics
• Quantify Shannon entropy in residual streams
• Unveil transformer workings via entropy metrics

Innovation

• Leverages information theory
• Quantifies Shannon entropy
• Analyzes vision transformers