DenseReviewer: A Screening Prioritisation Tool for Systematic Review based on Dense Retrieval

2025-02-05

AI Summary

To address the bottleneck of time- and labor-intensive initial screening of large-scale literature in medical systematic reviews, this paper proposes a dense-retrieval-driven active learning framework integrated with real-time human feedback. Methodologically, we build a dense passage retrieval (DPR) model using BERT-based semantic encoders, dynamically refine ranking strategies via interactive reviewer feedback, and implement a scalable web platform (using Flask) alongside a Python open-source library (built on PyTorch). Our key contribution lies in the first deep coupling of dense retrieval with online screening feedback, significantly improving screening efficiency: Recall@100 increases by 23.6%, and time-to-first-screening decreases by 41%. The toolkit is publicly released and has been deployed in real-world systematic review projects, enabling end-to-end acceleration of evidence identification.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Active Learning
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs

Abstract

Screening is a time-consuming and labour-intensive yet required task for medical systematic reviews, as tens of thousands of studies often need to be screened. Prioritising relevant studies to be screened allows downstream systematic review creation tasks to start earlier and save time. In previous work, we developed a dense retrieval method to prioritise relevant studies with reviewer feedback during the title and abstract screening stage. Our method outperforms previous active learning methods in both effectiveness and efficiency. In this demo, we extend this prior work by creating (1) a web-based screening tool that enables end-users to screen studies exploiting state-of-the-art methods and (2) a Python library that integrates models and feedback mechanisms and allows researchers to develop and demonstrate new active learning methods. We describe the tool's design and showcase how it can aid screening. The tool is available at https://densereviewer.ielab.io. The source code is also open sourced at https://github.com/ielab/densereviewer.

Problem

• Prioritising studies for medical systematic reviews
• Developing a dense retrieval method for screening
• Creating a web-based tool and Python library

Innovation

• Dense retrieval for study prioritization
• Web-based screening tool integration
• Open-source Python library development