Agentic systems for breast cancer treatment recommendations

2026-07-13

AI Summary

This study addresses the insufficient reliability of large language models in complex breast cancer treatment decision-making by proposing a novel paradigm that integrates a multi-agent architecture, autonomous sub-agent generation, and a fact-checking mechanism. To enable fine-grained and objective evaluation, the authors introduce an Asymmetric Information-based Rating Generation (AIRG) method. Leveraging foundation models such as Claude Opus 4.8 and incorporating tool use with clinical knowledge verification, multiple agent systems are systematically constructed and evaluated. The best-performing configuration, the D&C+SA pipeline, achieves a composite score of 0.594 ± 0.025 on real-world clinical cases, indicating moderate clinical relevance. However, the presence of critical errors underscores that the system is not yet suitable for unsupervised deployment in clinical settings.

Abstract

Large language models (LLMs) are increasingly being explored for clinical decision support, but their reliability in complex oncology treatment planning remains unclear. We evaluated agentic LLM systems for breast cancer treatment recommendation generation using 72 real clinical cases across stages I to IV and 1,147 case-specific rubrics generated through Asymmetric Information Rubric Generation (AIRG), in which the rubric generator had access to real clinical decisions unavailable to the evaluated models. Seven pipelines were compared, including single-LLM baselines, tool-augmented systems, and multi-agent architectures with fact checking and autonomous subagent spawning. The best-performing configuration, Claude Opus 4.8 with the D&C+SA pipeline, achieved a global score of 0.594 $\pm$ 0.025. Tool use and increased agent autonomy had mixed effects, improving performance in some settings but degrading it in others. Performance varied by clinical domain and disease stage, and oncologist-led error analysis revealed persistent clinically relevant failures, including incorrect or missing recommendations, flawed justifications, citation errors, outdated claims, and overconfidence. These findings suggest that agentic LLM systems can generate clinically relevant breast cancer recommendations, but remain insufficient for unsupervised clinical use.

Problem

• agentic systems
• breast cancer treatment
• clinical decision support
• large language models
• treatment recommendation

Innovation

• agentic LLM systems
• Asymmetric Information Rubric Generation
• multi-agent architecture
• clinical decision support
• breast cancer treatment recommendation