On the Reproducibility of Learned Sparse Retrieval Adaptations for Long Documents

2025-03-31

AI Summary

This paper addresses the high latency and computational overhead of Learning-to-Retrieve Sparse (LSR) methods in long-document retrieval, systematically investigating their reproducibility and adaptation mechanisms. We propose a segment-aggregation-based LSR optimization framework; ablation studies reveal that the first segment dominates retrieval effectiveness and underscore the critical role of global context modeling for long-document adaptation. Building upon ExactSDM and SoftSDM, we introduce several post-processing enhancements—n-gram augmentation, proximity-weighted scoring, representation fine-tuning, and multi-signal fusion—to significantly improve paragraph-level relevance modeling. Experiments demonstrate both effectiveness and reproducibility, and uncover a consistent positional sensitivity pattern: early segments exert disproportionate influence on final retrieval scores. All code and comprehensive experimental results are publicly released.

Technology Category

• Search and Optimization: Learning to Search
• Natural Language Processing: Summarization
• Machine Learning: Learning Preferences or Rankings

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs

Abstract

Document retrieval is one of the most challenging tasks in Information Retrieval. It requires handling longer contexts, often resulting in higher query latency and increased computational overhead. Recently, Learned Sparse Retrieval (LSR) has emerged as a promising approach to address these challenges. Some have proposed adapting the LSR approach to longer documents by aggregating segmented document using different post-hoc methods, including n-grams and proximity scores, adjusting representations, and learning to ensemble all signals. In this study, we aim to reproduce and examine the mechanisms of adapting LSR for long documents. Our reproducibility experiments confirmed the importance of specific segments, with the first segment consistently dominating document retrieval performance. Furthermore, We re-evaluate recently proposed methods -- ExactSDM and SoftSDM -- across varying document lengths, from short (up to 2 segments) to longer (3+ segments). We also designed multiple analyses to probe the reproduced methods and shed light on the impact of global information on adapting LSR to longer contexts. The complete code and implementation for this project is available at: https://github.com/lionisakis/Reproducibilitiy-lsr-long.

Problem

• Examining reproducibility of Learned Sparse Retrieval for long documents
• Evaluating segment dominance in long document retrieval performance
• Analyzing global information impact on LSR adaptation

Innovation

• Adapting LSR with segmented document aggregation
• Evaluating ExactSDM and SoftSDM methods
• Analyzing global information impact on LSR