Graphical Design of Interpretable Architectures

2026-08-19

AI Summary

本文针对AI架构设计中的表示问题，提出了一种基于Penrose张量符号的图形化方法，该方法能提供全局视图并直接映射到PyTorch代码。

Abstract

Designing, implementing, and comparing interpretable architectures requires a formal language to represent them. The most common representations fall short in one of two ways. Symbolic equations give no global view of an architecture at a glance. Probabilistic graphical models and flowcharts do not describe actual tensor manipulations, thus hiding key insights and limiting reproducibility. To close this gap, we introduce a graphical notation for designing interpretable AI architectures, adapted from Penrose tensor notation. This graphical notation gives a global view of an architecture and maps one to one onto PyTorch einsum code. We first use this notation to describe architectures that are interpretable by construction, including concept bottlenecks, sparse probes, prototype networks, neural additive models, and mixtures of linear models. We then diagram the key architectural components of Steerling-8B, a frontier interpretable language model. The diagram yields global insights into the architecture (e.g., showing that Steerling is a residual model), a geometric interpretation of each individual operation, and a direct translation into 33 lines of PyTorch code.

Problem

• interpretable AI architectures
• formal language
• tensor manipulations
• reproducibility

Innovation

• graphical notation
• interpretable AI architectures
• Penrose tensor notation
• PyTorch einsum code