Locating acts of mechanistic reasoning in student team conversations with mechanistic machine learning

2026-04-23

AI Summary

This study addresses the ongoing challenge of effectively identifying mechanistic reasoning segments in student team dialogues within STEM education research. To this end, the authors propose an intrinsically interpretable machine learning model that dynamically predicts the likelihood of mechanistic reasoning by integrating features from both individual utterances and group-level interactions. The model incorporates task-oriented inductive biases through a carefully designed probabilistic graphical structure combined with mechanistic learning techniques, substantially enhancing its generalization to unseen students and novel discussion contexts. Experimental results demonstrate that the proposed approach significantly outperforms baseline models, offering educational researchers a practical and interpretable analytical tool for examining collaborative reasoning processes.

Technology Category

• Machine Learning: Statistical Relational/Logic Learning
• Knowledge Representation and Reasoning: Reasoning with Beliefs
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

STEM education researchers are often interested in identifying moments of students' mechanistic reasoning for deeper analysis, but have limited capacity to search through many team conversation transcripts to find segments with a high concentration of such reasoning. We offer a solution in the form of an interpretable machine learning model that outputs time-varying probabilities that individual students are engaging in acts of mechanistic reasoning, leveraging evidence from their own utterances as well as contributions from the rest of the group. Using the toolkit of intentionally-designed probabilistic models, we introduce a specific inductive bias that steers the probabilistic dynamics toward desired, domain-aligned behavior. Experiments compare trained models with and without the inductive bias components, investigating whether their presence improves the desired model behavior on transcripts involving never-before-seen students and a novel discussion context. Our results show that the inductive bias improves generalization -- supporting the claim that interpretability is built into the model for this task rather than imposed post hoc. We conclude with practical recommendations for STEM education researchers seeking to adopt the tool and for ML researchers aiming to extend the model's design. Overall, we hope this work encourages the development of mechanistically interpretable models that are understandable and controllable for both end users and model designers in STEM education research.

Problem

• mechanistic reasoning
• student team conversations
• STEM education
• conversation transcripts
• reasoning detection

Innovation

• mechanistic reasoning
• interpretable machine learning
• inductive bias
• probabilistic modeling
• STEM education