Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agents

2024-05-05

arXiv.org

AI Summary

Large language model (LLM)-driven virtual physicians often lack robust diagnostic capabilities in real-world clinical settings due to insufficient domain-specific reasoning and limited exposure to diverse, high-fidelity medical scenarios. Method: We propose MedAgent-Zero—a knowledge-driven, closed-loop evolutionary multi-agent virtual hospital simulation framework. It integrates disease dynamics modeling, a structured medical knowledge base, and collaborative multi-agent orchestration to simulate end-to-end clinical workflows involving patient, nurse, and physician agents. The physician agent autonomously refines its clinical decision-making policy through reinforcement learning on ~10,000 synthetic cases with success/failure feedback, then transfers the learned capabilities to real-world evaluation. Contribution/Results: On the respiratory-disease subset of MedQA, MedAgent-Zero achieves 93.06% accuracy—setting a new state-of-the-art—and demonstrates significantly improved generalization over existing LLM-based diagnostic approaches.

Technology Category

• Multiagent Systems: Multiagent Learning
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

In this paper, we introduce a simulacrum of hospital called Agent Hospital that simulates the entire process of treating illness. All patients, nurses, and doctors are autonomous agents powered by large language models (LLMs). Our central goal is to enable a doctor agent to learn how to treat illness within the simulacrum. To do so, we propose a method called MedAgent-Zero. As the simulacrum can simulate disease onset and progression based on knowledge bases and LLMs, doctor agents can keep accumulating experience from both successful and unsuccessful cases. Simulation experiments show that the treatment performance of doctor agents consistently improves on various tasks. More interestingly, the knowledge the doctor agents have acquired in Agent Hospital is applicable to real-world medicare benchmarks. After treating around ten thousand patients (real-world doctors may take over two years), the evolved doctor agent achieves a state-of-the-art accuracy of 93.06% on a subset of the MedQA dataset that covers major respiratory diseases. This work paves the way for advancing the applications of LLM-powered agent techniques in medical scenarios.

Problem

• Virtual Doctors
• Diagnostic Ability
• Simulation Environment

Innovation

• Agent Hospital
• MedAgent-Zero
• AI-controlled healthcare agents