Stochastic Constrained Test Assembly for AI-Enabled Assessment Systems

2026-07-10

AI Summary

This study addresses the challenge in AI-based assessment systems where dynamic item banks contain newly introduced items with uncertain parameters, rendering traditional static test assembly inadequate for simultaneously ensuring measurement precision, adherence to content blueprint constraints, item bank sustainability, and efficient calibration of new items. To overcome this, the authors formulate linear test assembly as a constrained sequential decision-making problem and propose a Test-level Stochastic Constraint Hybrid (SCH) framework. SCH uniquely incorporates items with uncertain parameters directly into the assembly process, extending multi-armed bandit methodology from the item level to the test level by using Fisher information as the reward signal. Experimental results demonstrate that SCH significantly improves measurement accuracy, accelerates new-item calibration, and maintains item bank health while satisfying content constraints, outperforming six existing methods.

Abstract

Test assembly, the process of constructing a complete test form from an item pool subject to blueprint constraints, has traditionally been treated as a static optimization problem. In AI-enabled assessment environments, however, item pools evolve continuously as newly generated items enter with uncertain psychometric parameters, and delivery is on demand. These conditions make test assembly a sequential decision-making problem under uncertainty: which form should be deployed now, given current but incomplete knowledge of item quality, to simultaneously maximize measurement precision, satisfy content-blueprint constraints, maintain pool sustainability, and accelerate calibration of uncertain new items? This paper proposes the Stochastic Constrained Hybrid (SCH) framework as a principled answer to this question. SCH recasts form-level assembly as a multi-armed bandit (MAB) problem with Fisher information as the reward, extending recent item-level approaches in computerized adaptive testing (CAT) to the form-level setting. A simulation study comparing six test assembly methods is also presented. The main contribution of this paper is a framework for incorporating items with uncertain parameters into the automatic test assembly process for linear test forms.

Problem

• stochastic test assembly
• AI-enabled assessment
• uncertain item parameters
• blueprint constraints
• sequential decision-making

Innovation

• Stochastic Constrained Hybrid
• multi-armed bandit
• test assembly
• Fisher information
• AI-enabled assessment