PsyScore: A Psychometrically-Aware Framework for Trait-Adaptive Essay Scoring and ZPD-Scaffolded Feedback

2026-06-18

AI Summary

This study addresses critical limitations in existing automated essay scoring systems—namely, the disconnect between scoring and feedback, insufficient model interpretability, and non-adaptive feedback generation. To overcome these issues, the authors propose a unified framework that integrates Item Response Theory (IRT) with the Zone of Proximal Development (ZPD) concept, leveraging a shared latent proficiency representation to jointly optimize accurate scoring and personalized instructional feedback. Methodologically, they develop a neural IRT scorer based on the Generalized Partial Credit Model (GPCM) and introduce a multi-agent, proficiency-aware feedback generation mechanism guided by a multi-perspective evaluation strategy. Experimental results on the ASAP++ dataset demonstrate that the proposed system achieves state-of-the-art scoring performance while producing hierarchical, interpretable feedback aligned with educational objectives.

Abstract

Effective Automated Essay Scoring (AES) are expected to support both reliable assessment and actionable instructional feedback. However, existing approaches often treat scoring and feedback as separate components: neural scoring models provide limited interpretability, while Large Language Model (LLM)-based feedback is typically insensitive to learners proficiency levels. To address this fragmentation, this work proposes PsyScore, a psychometrically-aware framework that integrates diagnostic assessment with instructional scaffolding through a shared latent ability representation. PsyScore comprises three key modules: a Trait-Adaptive Neural IRT Scorer that incorporates the Graded Partial Credit Model (GPCM) into a neural architecture, enabling the precise estimation of student ability while maintaining psychometric interpretability, a ZPD-Scaffolded Feedback Generator, which conditions multi-agent feedback strategies on the diagnosed ability parameter to adapt instructional focus across different proficiency levels, and a Multi-Perspective Feedback Evaluation Strategy that assesses feedback quality via pairwise preference judgements and student revision simulations. Experiments on the ASAP++ dataset demonstrate that PsyScore achieves competitive scoring performance while providing more pedagogically aligned feedback.

Problem

• Automated Essay Scoring
• Instructional Feedback
• Proficiency Adaptation
• Psychometric Interpretability
• Formative Assessment

Innovation

• Trait-Adaptive Scoring
• Psychometrically-Aware Framework
• ZPD-Scaffolded Feedback
• Neural IRT
• Multi-Agent Feedback