Understanding Student Effort Using Response-Time Propensities During Problem Solving

2026-05-09

AI Summary

This study addresses the limitation of traditional metrics—such as task duration—in distinguishing whether students’ prolonged response times stem from deep cognitive engagement or item difficulty. It introduces response time tendency, defined as step-level response speed adjusted for item difficulty, as a stable individual difference measure. By integrating knowledge tracing with temporal process modeling, the authors employ a hierarchical statistical model to estimate effort-related traits at both student and knowledge component levels from classroom log data and examine their association with learning efficiency. Analysis of data from 794 students reveals that slower response tendencies significantly enhance learning efficiency among high-ability students, whereas no such benefit—and even a negative association—is observed for low-ability students. Notably, this signal exhibits the strongest diagnostic value early in practice, offering a novel approach for early detection of low-engagement states.

Abstract

Adaptive learning systems can produce substantial learning gains, yet many students engage for too brief or too superficial a period to benefit. A central obstacle is measuring effort. Effort during multi-step problem solving is rarely directly observed, and common log-based proxies, such as time on task, cannot distinguish between a student working carefully and a student encountering a harder problem. We examine step-to-step response time as a scalable effort signal by modeling trait-like differences in students' typical response timing during tutoring (while adjusting for skill difficulty). Using step-level logs from eight classroom deployments of algebra tutoring systems (2020 to 2023) across six U.S. schools (794 students), we estimate student- and knowledge-component-level propensities using hierarchical models and relate them to learning efficiency, defined as performance improvement per completed solution step. Response-time propensities show moderate to strong stability within students, supporting their use as an individual differences measure beyond correctness. At the same time, their relationship to learning is not uniform but conditional on the learner and context. Slower propensities predict greater learning efficiency for higher-proficiency students, consistent with constructive processing, whereas for lower-proficiency students, slower propensities are weakly related or even negative, consistent with unproductive struggle or idling. These associations are strongest early in practice sequences and attenuate later in the class period, highlighting an actionable window for detecting emerging disengagement and low persistence. Overall, response-time propensities provide a practical way to incorporate temporal process data into learner models and to target adaptive supports when effort is most diagnostic.

Problem

• student effort
• response time
• adaptive learning
• problem solving
• learning efficiency

Innovation

• response-time propensities
• effort measurement
• adaptive learning
• hierarchical modeling
• learning efficiency