🤖 AI Summary
This work addresses the critical threat of backdoor attacks to language model security, which existing interpretability-based detection methods struggle to identify effectively. The study proposes a novel backdoor detection mechanism grounded in activation divergence, revealing for the first time that backdoor triggers manifest as directional shifts in activation patterns rather than sparse feature activations. Through systematic analysis of multi-layer representations in SmolLM2-360M under both LoRA and full-rank fine-tuning settings, the authors compare the backdoor identification capabilities of Crosscoders and differential sparse autoencoders (Diff-SAE). Experimental results demonstrate that Diff-SAE achieves a backdoor isolation score of 0.40, perfect precision of 1.0, and zero false positives across multiple configurations—substantially outperforming Crosscoders, which attain a backdoor isolation score below 0.02. These findings underscore the fundamental advantage of differential representations in isolating backdoor behaviors.
📝 Abstract
Backdoor attacks on language models pose a significant threat to AI safety, where models behave normally on most inputs but exhibit harmful behavior when triggered by specific patterns. Detecting such backdoors through mechanistic interpretability remains an open challenge. We investigate two sparse autoencoder architectures -- Crosscoders and Differential SAEs (Diff-SAE) -- for isolating backdoor-related features in fine-tuned models. Using a controlled SQL injection backdoor triggered by year-based context ("2024" triggers vulnerable code, "2023" triggers safe code), we evaluate both approaches across LoRA and full-rank fine-tuning regimes on SmolLM2-360M. We find that Diff-SAE consistently and substantially outperforms Crosscoders for backdoor isolation. Diff-SAE achieves a Backdoor Isolation Score (BIS) of 0.40 with perfect precision (1.0) and zero false positive rate across most experimental conditions, while Crosscoders fail almost entirely with BIS below 0.02 in most cases. This performance gap holds across multiple transformer layers (14, 18, 22, 26) and both fine-tuning regimes, with full-rank fine-tuning producing particularly clean backdoor signals. Our results suggest that backdoors manifest as directional activation shifts rather than sparse feature activations, making difference-based representations fundamentally more effective for detection. These findings have important implications for AI safety monitoring and the development of interpretability tools for detecting model manipulation.