Tracing Query Expansion Effects through Sparse Autoencoder Features

2026-09-07

AI Summary

本文通过稀疏自动编码器特征追踪查询扩展效果，分析其如何改变密集检索器内部机制，并提出一种轻量级的精确检索行为调节方法。

Abstract

Query expansion (QE) is a critical technique in information retrieval that enriches underspecified queries with additional textual context. However, its effect is often unreliable in modern dense retrieval, especially for strong off-the-shelf retrievers without retraining. Existing studies mainly examine expansion quality, semantic drift, or retrieval outcomes, but rarely explain how QE changes dense retrievers internally. In this work, we trace QE effects through sparse autoencoder (SAE) features. Using paired original and expanded queries, we decompose layer-wise retriever representations into sparse latent activations, identify QE-related latents from expansion-induced activation shifts, and interpret them with natural-language descriptions and retrieval cases. Our analysis shows that effective QE induces layer-concentrated changes in sparse latents aligned with retrieval intent and entity attributes, rather than only perturbing final query embeddings. SAE-based activation steering further validates these latents improve retrieval more consistently than random interventions or vanilla QE across four benchmarks, suggesting that SAEs can explain QE effects and offer a lightweight option for precise retrieval behavior modulation without query rewriting or retriever fine-tuning.

Problem

• Query Expansion
• Dense Retrieval
• Sparse Autoencoder
• Retriever Representations
• Activation Shifts

Innovation

• Sparse Autoencoder
• Query Expansion
• Dense Retrieval
• Latent Activation
• Activation Steering