A Systematic Comparison of Multilingual Interpretability Methods Reveals Anisotropy-Driven Failures

2026-09-04

AI Summary

本文通过对比四种跨语言共享度量方法，揭示了由于向量空间中的各向异性导致的测量差异，并推荐ILO作为主要的共享度量指标。

Abstract

Multilingual language models develop shared cross-lingual representations, and various interpretability methods claim to quantify this sharing. These methods have been developed largely in isolation, and when they disagree, it is unclear whether the disagreement reflects a property of the model or an artifact of the measurement. We compare four sharing metrics (CKA, ANC, GMM dominance per token, and ILO) across 21 base models from five families (125M-14B parameters) and correlate each with cross-lingual transfer on five downstream tasks. We find that the metrics differ in their quantification of cross-lingual sharing in these models and suggest that the disagreement traces to anisotropy, the tendency of representations to cluster in a narrow cone of the embedding space. Only ILO's correlation with cross-lingual transfer (Spearman's $ρ= 0.90$) survives controls for model size, family, and per-task variation. We therefore recommend ILO as the primary sharing metric, to be reported alongside anisotropy diagnostics.

Problem

• multilingual language models
• interpretability methods
• cross-lingual sharing
• anisotropy
• metrics

Innovation

• ILO
• anisotropy
• cross-lingual sharing