Recurrence Is Not Enough: Causally Validating Multilingual SAE Translation Features in Gemma 2 and 3

2026-09-04

AI Summary

研究通过验证多语言环境下SAE特征的因果有效性，发现仅特征重复出现不足以证明其跨语言一致性，但找到了一个在23种语言设置下一致有效的翻译启动特征。

Abstract

Sparse autoencoder (SAE) features are increasingly used to explain and steer language-model behavior, but it remains unclear whether a feature found in one language context plays the same causal role when processing prompts in another language. We study this question using translation-initiation features (Wu et al., 2026). We reproduce the SAE feature discovery method from Wu et al. in Gemma 2 and extend it to multilingual settings that vary prompt language, source language, and target language. We then test whether features that recur across settings affect translation behavior by amplifying or ablating their activations during inference. We also examine whether the method can be applied to Gemma 3. In both models, we observe an identical finding: although we can find more than 20 features that activate frequently across all discovery settings, causal validation shows that nearly all have small or inconsistent effects. In contrast, one feature -- Gemma 2's (L10, 5717) and Gemma 3's (L20, 2456) -- consistently improves COMET scores when amplified and degrades them when ablated across 23 language settings. These results show that feature recurrence can overstate cross-lingual transfer, while identifying a language-agnostic translation-initiation direction in Gemma 2 and Gemma 3.

Problem

• Sparse Autoencoder
• Causal Validation
• Multilingual Settings
• Translation Features
• Cross-lingual Transfer

Innovation

• Causal Validation
• Multilingual SAE Features
• Translation Initiation Feature