Interpretable Difficulty-Aware Knowledge Tracing in Tutor-Student Dialogues

2026-05-01

AI Summary

This work addresses the limitations of existing conversational knowledge tracing approaches, which often neglect item difficulty modeling and rely on opaque representations from large language models (LLMs), resulting in predictions that lack both accuracy and interpretability. To overcome these issues, we propose the first conversational knowledge tracing framework that integrates Item Response Theory (IRT) with LLMs. Our method explicitly models student ability and item difficulty at each dialogue turn and maps LLM outputs to interpretable IRT parameters, thereby enabling cognitively grounded and transparent predictions. Evaluated on two real-world teacher–student dialogue datasets, the proposed approach consistently outperforms current baselines, significantly enhancing the transparency, theoretical coherence, and predictive performance of personalized tutoring systems.

Abstract

Recent advances in large language models (LLMs) have led to the development of AI-powered tutoring systems that provide interactive support via dialogue. To enable these tutoring systems to provide personalized support, it is essential to assess student performance at each turn, motivating knowledge tracing (KT) in dialogue settings. However, existing dialogue-based KT approaches often ignore question difficulty modeling and rely on opaque latent representations from LLMs, hindering accurate and interpretable prediction. In this work, we propose an interpretable difficulty-aware conversational KT framework built upon LLMs, which explicitly models students' abilities and the difficulty of tutor-posed tasks at each turn. The framework incorporates the original textual question and the next tutor-posed task to estimate the student's knowledge state and the difficulty of the upcoming turn. Furthermore, it integrates Item Response Theory to map LLM's outputs into student ability and question difficulty parameters, enabling interpretable prediction of student performance grounded in cognitive theories of learning. We evaluate the framework on two tutor-student dialogue datasets. Both quantitative and qualitative results show that our framework outperforms existing KT baselines, meanwhile generating interpretable outputs consistent with cognitive theory.

Problem

• knowledge tracing
• question difficulty
• interpretability
• tutor-student dialogue
• large language models

Innovation

• interpretable knowledge tracing
• difficulty-aware modeling
• Item Response Theory
• large language models
• tutor-student dialogue