Exploring Sparse Autoencoders in Text-Based Causal Confounding Adjustment

2026-09-01

AI Summary

研究使用稀疏自动编码器在基于文本的因果混淆调整中寻找平衡，通过迭代选择最小特征集来解决高维表示与有限样本重叠之间的矛盾。

Abstract

In many settings, studying causal questions based on text data requires adjusting for confounding information within texts. Yet there is a tradeoff in constructing text representations for adjustment: they must be sufficiently large and/or dense to preserve the confounding variables necessary for unbiased effect estimation, but sufficiently small and/or sparse to satisfy finite-sample overlap and yield low-variance estimates. To address this tradeoff, we turn to sparse autoencoders (SAEs), and propose a novel causal adjustment pipeline that iteratively selects a minimal set of SAE features via conditional independence tests. We find that SAE representations achieve better adjustments (lower bias and and higher coverage) than alternative representations in standard semi-synthetic evaluations with binary confounders, and their interpretability offers opportunities for falsification. We also introduce a more realistic semi-synthetic evaluation that uses multi-label data as the unobserved confounders and find off-the-shelf adjustment methods require increased investigation for these more complex settings. Code: https://github.com/mianzg/sae-text-confounder

Problem

• text-based causal confounding
• sparse autoencoders
• confounding variables
• bias
• variance

Innovation

• Sparse Autoencoders
• Causal Confounding Adjustment
• Conditional Independence Tests
• Interpretability