Towards Intrinsic Interpretability of Large Language Models:A Survey of Design Principles and Architectures

2026-04-17

AI Summary

The "black-box" nature of large language models hinders their trustworthy and secure deployment. This work presents a systematic survey of intrinsic interpretability research and, for the first time, proposes five core design paradigms: functional transparency, concept alignment, decomposable representations, explicit modularity, and latent sparsity induction. By establishing a unified taxonomic framework that integrates multiple technical approaches, the study clarifies the evolving research landscape, identifies key challenges, and outlines promising future directions. The resulting synthesis offers both theoretical foundations and architectural guidance for developing more interpretable and trustworthy large language models.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Interpretability, Explainability, and Transparency

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

While Large Language Models (LLMs) have achieved strong performance across many NLP tasks, their opaque internal mechanisms hinder trustworthiness and safe deployment. Existing surveys in explainable AI largely focus on post-hoc explanation methods that interpret trained models through external approximations. In contrast, intrinsic interpretability, which builds transparency directly into model architectures and computations, has recently emerged as a promising alternative. This paper presents a systematic review of the recent advances in intrinsic interpretability for LLMs, categorizing existing approaches into five design paradigms: functional transparency, concept alignment, representational decomposability, explicit modularization, and latent sparsity induction. We further discuss open challenges and outline future research directions in this emerging field. The paper list is available at: https://github.com/PKU-PILLAR-Group/Survey-Intrinsic-Interpretability-of-LLMs.

Problem

• Large Language Models
• Intrinsic Interpretability
• Model Transparency
• Explainable AI
• Trustworthy AI

Innovation

• intrinsic interpretability
• large language models
• design paradigms
• transparent architectures
• concept alignment