ir_explain: a Python Library of Explainable IR Methods

2024-04-29

arXiv.org

AI Summary

Neural ranking models improve retrieval performance but severely compromise interpretability. To address this, we propose ExIR—the first modular, extensible, open-source framework for explainable information retrieval (ExIR), implemented in Python and supporting pointwise, pairwise, and listwise post-hoc explanation granularities. ExIR unifies over ten state-of-the-art explanation methods—including SHAP, LIME, attention visualization, gradient-based attribution, and axiomatic analysis—within a single, coherent architecture. It features deep integration with Pyserini and irdatasets, enabling one-click reproduction of benchmark experiments on MS MARCO and TREC collections. Designed for seamless cross-toolchain evaluation, ExIR significantly lowers the barrier to entry for ExIR research. Its plug-and-play design facilitates rapid method prototyping, comparative analysis, and reproducible evaluation. The framework has already been adopted as a standard development and evaluation platform by multiple research teams worldwide.

Technology Category

• Natural Language Processing: Information Extraction
• Machine Learning: Learning Preferences or Rankings
• Humans and AI: Explainable AI (XAI) for Human Understanding

Application Category

• Search and Retrieval-Augmented AI: Web search models and ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

While recent advancements in Neural Ranking Models have resulted in significant improvements over traditional statistical retrieval models, it is generally acknowledged that the use of large neural architectures and the application of complex language models in Information Retrieval (IR) have reduced the transparency of retrieval methods. Consequently, Explainability and Interpretability have emerged as important research topics in IR. Several axiomatic and post-hoc explanation methods, as well as approaches that attempt to be interpretable-by-design, have been proposed. This article presents irexplain, an open-source Python library that implements a variety of well-known techniques for Explainable IR (ExIR) within a common, extensible framework. irexplain supports the three standard categories of post-hoc explanations, namely pointwise, pairwise, and listwise explanations. The library is designed to make it easy to reproduce state-of-the-art ExIR baselines on standard test collections, as well as to explore new approaches to explaining IR models and methods. To facilitate adoption, irexplain is well-integrated with widely-used toolkits such as Pyserini and irdatasets.

Problem

• Lack of transparency in neural ranking models for IR
• Need for explainability and interpretability in IR methods
• Developing a unified library for Explainable IR techniques

Innovation

• Open-source Python library for Explainable IR
• Supports pointwise, pairwise, listwise explanations
• Integrates with Pyserini and irdatasets toolkits