Completing A Systematic Review in Hours instead of Months with Interactive AI Agents

2025-04-21

AI Summary

Systematic reviews (SRs) are critical for evidence-based medical decision-making, yet conventional SR production is labor-intensive and time-consuming—often requiring months—and existing automated approaches lack domain-specific adaptation and interpretability, compromising reliability. To address these challenges, we propose InsightAgent, an interactive AI agent that introduces semantic-driven literature chunking and a multi-agent collaboration framework, integrated with real-time expert feedback, interpretable visualization of reasoning trajectories, and a human-in-the-loop optimization interface. Empirical evaluation demonstrates that InsightAgent generates high-quality SRs within hours, achieving 79.7% of human-level abstract quality, accelerating the process by over 99%, and increasing user satisfaction by 34.4%. This work advances trustworthy, efficient, and auditable AI-augmented evidence synthesis, establishing a new paradigm for clinically grounded, human-centered systematic review automation.

Technology Category

• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Agent/AI Theories and Architectures
• Humans and AI: Planning and Decision Support for Human-Machine Teams

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Systematic reviews (SRs) are vital for evidence-based practice in high stakes disciplines, such as healthcare, but are often impeded by intensive labors and lengthy processes that can take months to complete. Due to the high demand for domain expertise, existing automatic summarization methods fail to accurately identify relevant studies and generate high-quality summaries. To that end, we introduce InsightAgent, a human-centered interactive AI agent powered by large language models that revolutionize this workflow. InsightAgent partitions a large literature corpus based on semantics and employs a multi-agent design for more focused processing of literature, leading to significant improvement in the quality of generated SRs. InsightAgent also provides intuitive visualizations of the corpus and agent trajectories, allowing users to effortlessly monitor the actions of the agent and provide real-time feedback based on their expertise. Our user studies with 9 medical professionals demonstrate that the visualization and interaction mechanisms can effectively improve the quality of synthesized SRs by 27.2%, reaching 79.7% of human-written quality. At the same time, user satisfaction is improved by 34.4%. With InsightAgent, it only takes a clinician about 1.5 hours, rather than months, to complete a high-quality systematic review.

Problem

• Accelerate systematic reviews from months to hours using AI
• Improve accuracy of study identification and summary generation
• Enhance user interaction and visualization for real-time feedback

Innovation

• Interactive AI agent powered by large language models
• Semantic partitioning and multi-agent literature processing
• Real-time visualizations and user feedback integration