RadOnc-GPT: An Autonomous LLM Agent for Real-Time Patient Outcomes Labeling at Scale

2025-09-29

AI Summary

Manual annotation constrains the scale, accuracy, and timeliness of prognostic research in radiation oncology. To address this, we propose an autonomous large language model (LLM) agent framework that enables real-time, large-scale extraction and structured annotation of patient-specific information from electronic health records for critical clinical outcomes—including osteoradionecrosis (ORN) of the jaw and tumor recurrence. Methodologically, the framework employs a hierarchical verification mechanism integrating structured data retrieval with multi-step reasoning over unstructured clinical text, augmented by iterative evidence evaluation to ensure reasoning reliability. Evaluated on head-and-neck and prostate cancer cohorts, it achieves high accuracy in demographic and radiotherapy parameter extraction, and >92% accuracy in ORN and recurrence classification. This significantly enhances annotation scalability and temporal responsiveness, establishing the first verifiable, closed-loop paradigm for autonomous medical reasoning in complex clinical settings.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Manual labeling limits the scale, accuracy, and timeliness of patient outcomes research in radiation oncology. We present RadOnc-GPT, an autonomous large language model (LLM)-based agent capable of independently retrieving patient-specific information, iteratively assessing evidence, and returning structured outcomes. Our evaluation explicitly validates RadOnc-GPT across two clearly defined tiers of increasing complexity: (1) a structured quality assurance (QA) tier, assessing the accurate retrieval of demographic and radiotherapy treatment plan details, followed by (2) a complex clinical outcomes labeling tier involving determination of mandibular osteoradionecrosis (ORN) in head-and-neck cancer patients and detection of cancer recurrence in independent prostate and head-and-neck cancer cohorts requiring combined interpretation of structured and unstructured patient data. The QA tier establishes foundational trust in structured-data retrieval, a critical prerequisite for successful complex clinical outcome labeling.

Problem

• Automating patient outcomes labeling in radiation oncology
• Overcoming manual labeling limitations in scale and accuracy
• Validating autonomous LLM agent for complex clinical assessments

Innovation

• Autonomous LLM agent for patient outcomes labeling
• Retrieves patient data and assesses evidence iteratively
• Validated through structured QA and clinical outcomes tiers