Surrogate Fidelity: When Can Open LLMs Explain Closed Ones?

2026-06-30

AI Summary

This study addresses the challenge of evaluating whether open-source language models can effectively serve as proxies to interpret the behavior of closed-source large language models when internal access is unavailable. Employing API-compatible methods—including log-odds probing, leave-one-out attribution, attention analysis, and input ablation—the authors conduct cross-model comparisons across 11 models spanning four major families: Llama, Qwen, GPT, and Gemini. Their findings reveal that predictive consistency substantially exceeds attribution consistency across model pairs. While white-box signals exhibit stability, they often fail to accurately reflect underlying causal mechanisms; in contrast, black-box input ablation more reliably captures the attribution behavior of closed-source models. These results uncover an “access-effectiveness inversion,” demonstrating that alignment in predictions alone is insufficient to support the transferability of mechanistic interpretations.

Abstract

Mechanistic interpretability (MI) requires full access to model internals, yet the APIs for most widely deployed language models at best expose log-probabilities over output tokens. This creates a surrogate problem: when do measurements made on open models allow us to make claims about a closed model? We evaluate surrogate fidelity at the prediction, attribution, and representation levels. For binary classification tasks, log-odds provide an API-compatible scalar readout of the model's representation space, and leave-one-out attributions provide insight into model behavior. Across eleven models spanning four families (Llama, Qwen, GPT, and Gemini), we find that prediction fidelity substantially overstates attribution fidelity: models that agree on what the answer is often disagree on why. We document an access-validity inversion: white-box signals like attention patterns and perturbation magnitudes are highly stable across models but only weakly predictive of causal attributions, which black-box input ablations capture by design. Mechanistic insight does not automatically transfer to closed targets, and prediction-level agreement is insufficient to warrant such transfer. Code and results are available at https://github.com/facebookresearch/surrogate.

Problem

• surrogate fidelity
• mechanistic interpretability
• closed language models
• attribution
• representation

Innovation

• Surrogate Fidelity
• Mechanistic Interpretability
• Model Attribution
• Black-box Ablation
• Access-Validity Inversion