A Representation-Learning Item Response Model for Identifying Behaviorally Important Actions in PIAAC Process Data

2026-08-17

AI Summary

This study addresses the challenge of identifying critical behaviors within complex, noisy process data by proposing a novel framework integrating representation learning with Bayesian Item Response Theory. The approach combines action-level and temporal context modeling while incorporating spike-and-slab priors to enable context-aware sparse association analysis. Empirical validation using PIAAC data demonstrates that the model accurately identifies sparse sets of key behaviors correlated with task performance, effectively quantifies uncertainty, and reveals variations in behavioral information distribution across different items. Consequently, this work establishes an interpretable paradigm for process mining, offering a robust solution for extracting meaningful behavioral signals from intricate sequential data in educational and cognitive assessments.

Abstract

Problem-solving log process data from computer-based assessments provide detailed information about how respondents approach and complete tasks. However, the resulting action sequences are complex and noisy, making it difficult to identify specific behaviors associated with successful performance. This paper proposes a representation-learning item response modeling (IRT) framework for identifying behaviorally important actions while accounting for respondent proficiency and item-level differences. Raw log sequences and timing information are first transformed into action representations that incorporate the hierarchical structure of action labels and the sequential and temporal context in which each action occurs. These respondent-specific representations are then entered as covariates in an extended IRT model, with spike-and-slab priors used to identify action-item combinations associated with response accuracy. The framework therefore evaluates actions contextually rather than as simple occurrence indicators and provides posterior uncertainty for their associations with performance. We apply the approach to problem-solving process data from the OECD Programme for the International Assessment of Adult Competencies (PIAAC). The analysis identifies a sparse set of actions associated with successful and unsuccessful performance and reveals differences across items in where behavioral information occurs within the problem-solving process.

Problem

• Process Data
• Action Identification
• Problem-Solving
• PIAAC
• Behavioral Analysis

Innovation

• Representation Learning
• Item Response Theory
• Spike-and-Slab Priors
• Process Data
• Behavioral Identification