GLUScope: A Tool for Analyzing GLU Neurons in Transformer Language Models

2026-02-27

AI Summary

Existing interpretability tools struggle to analyze how the sign combinations of gating and input signals in gated activation functions—such as SwiGLU—affect neuron behavior. To address this gap, this work proposes GLUScope, the first open-source tool enabling systematic analysis of the four sign-based configurations (positive/negative gate × positive/negative input) in GLU neurons within Transformer models. By integrating activation sign decomposition, text example extraction, and interactive visualization, GLUScope reveals distinct semantic roles associated with each configuration. Empirical results demonstrate that these sign combinations correspond to markedly different linguistic functions, offering novel insights into the inner workings of gating mechanisms. The tool and its accompanying demo platform have been publicly released to foster further community research.

Technology Category

• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Machine Learning: Mixture of Experts (MoE)
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

We present GLUScope, an open-source tool for analyzing neurons in Transformer-based language models, intended for interpretability researchers. We focus on more recent models than previous tools do; specifically we consider gated activation functions such as SwiGLU. This introduces a new challenge: understanding positive activations is not enough. Instead, both the gate and the in activation of a neuron can be positive or negative, leading to four different possible sign combinations that in some cases have quite different functionalities. Accordingly, for any neuron, our tool shows text examples for each of the four sign combinations, and indicates how often each combination occurs. We describe examples of how our tool can lead to novel insights. A demo is available at https: //sjgerstner.github.io/gluscope.

Problem

• GLU neurons
• Transformer language models
• gated activation functions
• neuron interpretability
• SwiGLU

Innovation

• GLUScope
• SwiGLU
• gated activation
• neuron interpretability
• sign combinations