Tool or Tutor? Experimental evidence from AI deployment in cancer diagnosis

2025-02-23

AI Summary

Despite growing deployment of AI in healthcare, its dual roles—as a training “mentor” and real-time diagnostic “tool”—remain underexplored, particularly regarding synergistic effects on human performance in high-stakes clinical decision-making. Method: We conducted a randomized controlled field experiment with 334 medical students, comparing three interventions: AI-only training, AI-only real-time assistance, and integrated AI training + assistance (“mentor + tool”). Diagnostic accuracy was quantitatively assessed across standardized lung cancer case vignettes. Contribution/Results: The integrated condition significantly improved diagnostic accuracy—by an average of 12.6% over single-role conditions (p < 0.001), the first empirical demonstration of complementary learning enhancement and task support. This establishes a novel human-AI collaboration paradigm: AI not only augments immediate clinical decisions but also fosters long-term expert competency through pedagogical internalization. The findings provide critical empirical grounding and a design framework for systemic AI integration in high-risk medical domains.

Technology Category

• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Professionals increasingly use Artificial Intelligence (AI) to enhance their capabilities and assist with task execution. While prior research has examined these uses separately, their potential interaction remains underexplored. We propose that AI-driven training (tutor effect) and AI-assisted task completion (tool effect) can be complementary and test this hypothesis in the context of lung cancer diagnosis. In a field experiment with 334 medical students, we manipulated AI deployment in training, in practice, and in both. Our findings reveal that while AI-integrated training and AI assistance independently improved diagnostic performance, their combination yielded the highest accuracy. These results underscore AI's dual role in enhancing human performance through both learning and real-time support, offering insights into AI deployment in professional settings where human expertise remains essential.

Problem

• AI enhances cancer diagnosis accuracy
• AI combines training and task assistance
• AI improves human diagnostic performance

Innovation

• AI-driven training enhances learning
• AI-assisted task improves diagnostics
• Combined AI roles maximize accuracy