ProIQA: A Process-Based Framework for Fine-Grained Math Item Quality Assessment

2026-09-14

AI Summary

为解决自动生成题目教育价值保证难题，提出基于过程的数学题目质量评估框架ProIQA，利用大语言模型和图神经网络构建解题表示，有效评估题目质量。

Abstract

Automatic Item Generation (AIG) is pivotal for personalized education, yet guaranteeing the pedagogical value of generated items remains a bottleneck. Existing Item Quality Assessment (IQA) methods typically rely on unscalable manual reviews or shallow stem-based metrics, failing to capture the reasoning process required for mathematical problem-solving. To bridge this gap, this paper proposes Process-based Item Quality Assessment (ProIQA), a process-aware framework for fine-grained quality assessment of math items. We first formulate IQA across three heterogeneous dimensions, including knowledge concepts, difficulty, and disciplinary competencies, under a unified process-aware perspective. Based on this formulation, we construct a process-enhanced IQA resource by augmenting original item data with structured reasoning trees derived from raw solutions. Technically, ProIQA leverages Large Language Modelsto construct hierarchical reasoning trees and employs Graph Neural Networks (GNN) to encode their topological dependencies and procedural semantics. The resulting solving representation is fused with stem semantics through a dual-view (``Stem + Solving'') architecture, enabling comprehensive assessment across learning objectives. Extensive experiments on K12 mathematical datasets show that ProIQA effectively captures process-oriented features, offering a scalable data-driven solution for evaluating AIG outputs in intelligent education systems.

Problem

• Automatic Item Generation
• Item Quality Assessment
• Mathematical Problem-Solving
• Reasoning Process
• Personalized Education

Innovation

• Process-based Item Quality Assessment
• Large Language Models
• Graph Neural Networks
• reasoning trees
• dual-view architecture