Robustness of IR Models to Collection Growth

2026-08-24

AI Summary

研究探讨了信息检索模型在文档集合增长时的鲁棒性问题，通过合并两个主题重叠较少的集合来评估不同模型的表现，发现依赖多文档的模型(MDD)和不依赖多文档的模型(MDA)都存在性能下降。

Abstract

Information Retrieval (IR) systems seek to identify relevant documents within a collection. In practical applications, collections are dynamic, with documents frequently added. We argue that ideally, a retriever's effectiveness should not decrease when non-relevant documents are added to a collection. This study formalises this concept and empirically evaluates it by merging two collections with negligible topic overlap. We hypothesise that the way an IR model conditions its ranking on other documents in a collection (e.g., the IDF component in BM25 or contextual documents in listwise rerankers) plays an important role in its robustness to the addition of non-relevant documents. We broadly classify models as those that do not depend on other documents (Multi-Document-Agnostic, MDA) and those that do (Multi-Document-Dependent, MDD). Our results show that neither MDD nor MDA models are fully robust to the addition of non-relevant documents, as all models exhibit some performance degradation. Interestingly, among the models we test, MDA is more effective than MDD for retrieval, whereas MDD and MDA rerankers are equally effective.

Problem

• Information Retrieval
• Collection Growth
• Non-relevant Documents
• Retriever's Effectiveness

Innovation

• Robustness
• Collection Growth
• Multi-Document-Agnostic (MDA)
• Multi-Document-Dependent (MDD)
• Information Retrieval