Reversing the Lens: Using Explainable AI to Understand Human Expertise

2025-09-06

arXiv.org

AI Summary

This study investigates the structured evolutionary mechanisms underlying human problem-solving strategies in complex, real-world tasks—exemplified by particle accelerator tuning—where optimal solutions are often inaccessible and uncertainty is high. Method: We propose an explainable artificial intelligence (XAI)-based cognitive analysis framework that constructs behavioral graphs from operator interaction logs and applies graph-theoretic techniques—including community detection and hierarchical clustering—to quantitatively model and dynamically track the evolution of domain expertise. Contribution/Results: Our analysis uncovers a systematic shift in problem decomposition patterns from novice to expert: novices rely on linear, localized subtasks, whereas experts develop modular, hierarchically organized strategy graphs. The framework not only validates XAI’s efficacy and interpretability in cognitive science research but also establishes a novel paradigm for dissecting experiential learning mechanisms in non-optimal, high-uncertainty operational environments.

Technology Category

• Humans and AI: Explainable AI (XAI) for Human Understanding
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Reasoning under Uncertainty: Causality

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Both humans and machine learning models learn from experience, particularly in safety- and reliability-critical domains. While psychology seeks to understand human cognition, the field of Explainable AI (XAI) develops methods to interpret machine learning models. This study bridges these domains by applying computational tools from XAI to analyze human learning. We modeled human behavior during a complex real-world task -- tuning a particle accelerator -- by constructing graphs of operator subtasks. Applying techniques such as community detection and hierarchical clustering to archival operator data, we reveal how operators decompose the problem into simpler components and how these problem-solving structures evolve with expertise. Our findings illuminate how humans develop efficient strategies in the absence of globally optimal solutions, and demonstrate the utility of XAI-based methods for quantitatively studying human cognition.

Problem

• Applying XAI techniques to analyze human learning processes
• Modeling operator behavior during complex particle accelerator tuning
• Revealing how problem-solving strategies evolve with expertise

Innovation

• Using XAI tools to analyze human learning processes
• Modeling human behavior via operator subtask graphs
• Applying community detection to reveal problem-solving structures