Uncertainty-Aware Knowledge Tracing Models

2025-09-25

AI Summary

Knowledge tracing (KT) models often fail to detect students’ erroneous selections of distractors, leaving latent misconceptions undiagnosed. To address this, we introduce predictive uncertainty modeling into KT for the first time, leveraging probabilistic deep learning to quantify per-prediction confidence. Experiments demonstrate a statistically significant positive correlation between high uncertainty and model misclassification (p < 0.01), enabling effective identification of student cognitive biases. Our approach requires no additional annotations or pedagogical interventions, yielding interpretable instructional signals that support precise diagnosis and adaptive remediation—even under resource constraints. Key contributions are: (1) the first uncertainty-aware KT framework; (2) empirical validation that uncertainty serves as a reliable proxy for prediction errors; and (3) a novel, operationally viable analytical dimension for trustworthy educational AI—balancing reliability with practical deployability.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The main focus of research on Knowledge Tracing (KT) models is on model developments with the aim of improving predictive accuracy. Most of these models make the most incorrect predictions when students choose a distractor, leading to student errors going undetected. We present an approach to add new capabilities to KT models by capturing predictive uncertainty and demonstrate that a larger predictive uncertainty aligns with model incorrect predictions. We show that uncertainty in KT models is informative and that this signal would be pedagogically useful for application in an educational learning platform that can be used in a limited resource setting where understanding student ability is necessary.

Problem

• Modeling predictive uncertainty in knowledge tracing to detect student errors
• Improving pedagogical utility of KT models in resource-limited educational platforms
• Aligning high predictive uncertainty with incorrect predictions in knowledge tracing

Innovation

• Capturing predictive uncertainty in knowledge tracing models
• Linking larger uncertainty with model incorrect predictions
• Applying uncertainty signals in educational learning platforms