Confidence-Based Stopping Methods for Systematic Reviews

2026-06-13

AI Summary

Current technology-assisted systematic review stopping mechanisms predominantly target a predefined recall rate, often overlooking the sufficiency of information required for robust decision-making. This work proposes two confidence-based heuristic stopping strategies that dynamically assess whether the screened literature provides adequate evidence to support reliable conclusions, thereby replacing conventional recall-oriented approaches. Evaluated on standard datasets of diagnostic test accuracy systematic reviews, the proposed methods enable efficient dynamic stopping, substantially reducing the number of articles requiring screening while, in most cases, still yielding conclusions consistent with those derived from the full evidence set—thus better aligning with the practical demands of evidence-based decision-making.

Abstract

Technology Assisted Review stopping methods aim to ensure that no more documents are screened than necessary. Most existing approaches focus on achieving a target recall, which does not consider whether an information need has been met. This paper introduces two heuristic stopping methods that instead monitor whether screened documents contain enough information to make a decision. Evaluation on a standard dataset of Diagnostic Test Accuracy Systematic Reviews demonstrates that the proposed approaches substantially reduce the number of documents that need to be examined while, in the majority of cases, maintaining conclusions that are consistent with all evidence available.

Problem

• Systematic Reviews
• Stopping Methods
• Technology Assisted Review
• Information Need
• Decision Making

Innovation

• confidence-based stopping
• technology-assisted review
• systematic reviews
• decision-focused screening
• document screening efficiency