One Adapter Pair per Model: A Universal Activation Interface for Language Models

2026-08-10

AI Summary

Existing activation-space tools—such as probes, sparse autoencoders, and natural language interpreters—typically rely on hidden representations specific to individual language models, limiting their reusability across different architectures. This work proposes a Universal Activation Bus framework that maps activations from diverse models into a unified, dense shared space via lightweight linear encoder–decoder adapter pairs trained per compatible model. This approach achieves, for the first time, cross-model semantic alignment of activations and standardization of tool interfaces: new models can be integrated by adding only their corresponding adapters, without retraining existing interpretability tools. Experiments across five language models demonstrate that semantically related texts form consistent neighborhoods in the shared space, enabling successful reuse of various interpretation tools and supporting cross-model, intermediate-activation-driven prediction.

Abstract

Activation-based tools are usually tied to one model's native hidden space, requiring probes, sparse autoencoders, and natural-language interpreters to be rebuilt or rediscovered for each new language model. We present a Universal Activation Bus, a framework that provides a common activation interface across compatible language models. Using a small set of source models, we learn a shared dense space together with one lightweight linear encoder--decoder adapter pair per model. After source training, the interface is frozen; a new model joins by fitting only its adapter pair on unlabeled matched text. The resulting interface allows activation-based tools to be shared across connected models, including common probes and SAE features as well as access to an NLA originally trained for a different model. Across five models, semantically related texts form consistent neighborhoods in the shared space, and an onboarded model reuses these tools effectively without retraining them. We further show that an intermediate activation from one model can be used by another model's frozen upper layers to produce predictions. These results establish a stable, model-wise activation contract for reusable tools across compatible language models.

Problem

• activation-based tools
• language models
• model compatibility
• universal interface
• cross-model reuse

Innovation

• Universal Activation Bus
• adapter pair
• shared activation space
• cross-model tool reuse
• frozen interface