AiReview: An Open Platform for Accelerating Systematic Reviews with LLMs

2025-04-05

AI Summary

Existing systematic review screening tools lack transparent, reproducible, and user-controllable integration of large language models (LLMs), resulting in high manual burden and low efficiency. This paper introduces AiReview, the first open-source, end-to-end LLM-augmented framework for title/abstract screening in medical systematic reviews. It supports prompt engineering, lightweight fine-tuning, and plug-and-play multi-model interoperability. Built on React and FastAPI, its interactive web interface enables human-in-the-loop annotation and fully traceable workflow provenance. Empirical evaluation demonstrates significantly improved screening sensitivity and interpretability over conventional machine learning approaches. The platform’s source code and deployment artifacts are publicly released to ensure community reproducibility and extensibility. Its core contribution lies in the systematic, methodologically transparent integration of state-of-the-art LLMs into real-world systematic review practice—balancing automation efficacy with rigorous, auditable methodology.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Intelligent User Interfaces
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Systematic reviews are fundamental to evidence-based medicine. Creating one is time-consuming and labour-intensive, mainly due to the need to screen, or assess, many studies for inclusion in the review. Several tools have been developed to streamline this process, mostly relying on traditional machine learning methods. Large language models (LLMs) have shown potential in further accelerating the screening process. However, no tool currently allows end users to directly leverage LLMs for screening or facilitates systematic and transparent usage of LLM-assisted screening methods. This paper introduces (i) an extensible framework for applying LLMs to systematic review tasks, particularly title and abstract screening, and (ii) a web-based interface for LLM-assisted screening. Together, these elements form AiReview-a novel platform for LLM-assisted systematic review creation. AiReview is the first of its kind to bridge the gap between cutting-edge LLM-assisted screening methods and those that create medical systematic reviews. The tool is available at https://aireview.ielab.io. The source code is also open sourced at https://github.com/ielab/ai-review.

Problem

• Accelerating systematic reviews using LLMs for screening studies
• Providing an open platform for LLM-assisted medical review creation
• Bridging the gap between LLM methods and medical review practices

Innovation

• Extensible framework for LLM-assisted systematic reviews
• Web-based interface for LLM screening tasks
• Open platform integrating LLMs with medical reviews