Structuring versus Problematizing: How LLM-based Agents Scaffold Learning in Diagnostic Reasoning

2026-04-10

AI Summary

This study addresses the challenges students face in diagnostic reasoning—particularly susceptibility to cognitive biases such as premature closure and overreliance on heuristics, as well as limited strategy transferability—within a situated learning environment for pharmacy technician training. For the first time, it comparatively examines two theory-driven scaffolding dialogue strategies: structured and problematizing. An intelligent tutoring agent, integrating learning analytics and large language models, dynamically intervenes in learners’ trajectories. Findings indicate that both scaffolding approaches effectively promote the use of diagnostic strategies: structured scaffolding enhances the accuracy of active interaction, while problematizing scaffolding fosters constructive engagement. Notably, task complexity exerts a significantly stronger influence on performance than either prior knowledge or scaffolding type.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Supporting students in developing diagnostic reasoning is a key challenge across educational domains. Novices often face cognitive biases such as premature closure and over-reliance on heuristics, and they struggle to transfer diagnostic strategies to new cases. Scenario-based learning (SBL) enhanced by Learning Analytics (LA) and large language models (LLM) offers a promising approach by combining realistic case experiences with personalized scaffolding. Yet, how different scaffolding approaches shape reasoning processes remains insufficiently explored. This study introduces PharmaSim Switch, an SBL environment for pharmacy technician training, extended with an LA- and LLM-powered pharmacist agent that implements pedagogical conversations rooted in two theory-driven scaffolding approaches: \emph{structuring} and \emph{problematizing}, as well as a student learning trajectory. In a between-groups experiment, 63 vocational students completed a learning scenario, a near-transfer scenario, and a far-transfer scenario under one of the two scaffolding conditions. Results indicate that both scaffolding approaches were effective in supporting the use of diagnostic strategies. Performance outcomes were primarily influenced by scenario complexity rather than students'prior knowledge or the scaffolding approach used. The structuring approach was associated with more accurate Active and Interactive participation, whereas problematizing elicited more Constructive engagement. These findings underscore the value of combining scaffolding approaches when designing LA- and LLM-based systems to effectively foster diagnostic reasoning.

Problem

• diagnostic reasoning
• scaffolding
• scenario-based learning
• cognitive biases
• transfer

Innovation

• scaffolding
• large language models
• diagnostic reasoning
• scenario-based learning
• learning analytics