A constraints-based approach to fully interpretable neural networks for detecting learner behaviors

2025-04-10

AI Summary

To address the lack of interpretability in machine learning models for educational applications, this paper proposes a constraint-driven fully interpretable neural network designed specifically to detect “gaming-the-system” behaviors—i.e., strategic, non-learning-oriented student actions. Methodologically, it introduces the paradigm “interpretability-as-design-principle,” embedding domain-specific behavioral priors, enforcing sparsity constraints, and explicitly modeling decision paths to ensure network parameters carry clear educational semantics—thereby guaranteeing explanation fidelity and human comprehensibility. Experiments demonstrate that the model achieves expert-level performance in detection, learns patterns highly consistent with educational experts’ judgments, and supports real-time, faithful, and interpretable behavioral attribution. This work establishes a novel, deployable paradigm for interpretable AI in education and delivers a reusable architectural framework.

Technology Category

• Machine Learning: Transparent, Interpretable, Explainable ML
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Computer Vision: Interpretability, Explainability, and Transparency

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The increasing use of complex machine learning models in education has led to concerns about their interpretability, which in turn has spurred interest in developing explainability techniques that are both faithful to the model's inner workings and intelligible to human end-users. In this paper, we describe a novel approach to creating a neural-network-based behavior detection model that is interpretable by design. Our model is fully interpretable, meaning that the parameters we extract for our explanations have a clear interpretation, fully capture the model's learned knowledge about the learner behavior of interest, and can be used to create explanations that are both faithful and intelligible. We achieve this by implementing a series of constraints to the model that both simplify its inference process and bring it closer to a human conception of the task at hand. We train the model to detect gaming-the-system behavior, evaluate its performance on this task, and compare its learned patterns to those identified by human experts. Our results show that the model is successfully able to learn patterns indicative of gaming-the-system behavior while providing evidence for fully interpretable explanations. We discuss the implications of our approach and suggest ways to evaluate explainability using a human-grounded approach.

Problem

• Develop interpretable neural networks for education behavior detection
• Ensure model explanations are faithful and intelligible to humans
• Detect gaming-the-system behavior with human-aligned constraints

Innovation

• Constraints-based interpretable neural networks
• Human-conceived task simplification via constraints
• Fully interpretable parameters for behavior detection