Lost in a Single Vector: Improving Long-Document Retrieval with Chunk Evidence Aggregation

2026-06-17

AI Summary

This work addresses the performance degradation in dense retrieval of long documents, where full-document encoding dilutes critical information. The authors propose DICE, a method that preserves the standard single-query–single-document interface while independently encoding document chunks and aggregating them into a single vector to retain the strongest evidence signals. They introduce the Evidence Dilution Index (EDI) to quantify information loss in document representations and design a training-free, document-side chunk aggregation strategy. Evaluated on LongEmbed using frozen dense retrieval models, DICE substantially improves long-document retrieval: for passages longer than 4k tokens, Passkey and Needle task scores rise from 30.0/23.3 to 90.0/74.0, with 92.8% of samples exhibiting significantly reduced EDI.

Abstract

Dense retrieval ranks one query vector against one document vector. On long documents, this interface can fail when a short but decisive span is weakened during document encoding before ranking. We study this failure mode as document-side early compression and introduce the Evidence Dilution Index (EDI) to measure how far a document-level representation falls below the strongest chunk-level evidence within the same gold document. Guided by this view, we propose DICE (Document Inference via Chunk Evidence), a training-free document-side strategy that splits documents into chunks, encodes them independently with a frozen model, and aggregates them back into a single vector while preserving the standard one-query-one-document interface. On LongEmbed, DICE improves retrieval across four backbones, with the largest gains on slices beyond 4k tokens: for Dream, Passkey >4k rises from 30.0 to 90.0 and Needle >4k from 23.3 to 74.0. Across 12,779 filtered samples, DICE yields lower EDI than the single-vector baseline in 92.8% of cases. These results establish document-level encoding as a practical and underexplored lever for long-document retrieval.

Problem

• long-document retrieval
• dense retrieval
• evidence dilution
• document encoding
• chunk evidence

Innovation

• dense retrieval
• long-document retrieval
• chunk evidence aggregation
• Evidence Dilution Index
• training-free method