PEOAT: Personalization-Guided Evolutionary Question Assembly for One-Shot Adaptive Testing

2025-11-29

AI Summary

Traditional computerized adaptive testing (CAT) relies on iterative item selection, suffering from poor real-time performance, high cognitive interference, and elevated administration costs—particularly in large-scale or sensitive psychological assessments. To address these limitations, this work introduces One-shot Adaptive Testing (OAT): a novel task requiring the generation of a fixed-length, optimal item subset for each examinee in a single decision step. We establish the first formal OAT framework and propose a personalized-guided evolutionary algorithm integrating ability–difficulty matching modeling, cognition-aware mutation, and diversity-preserving mechanisms. Our method employs multi-strategy initialization, pattern-preserving crossover, and environment-aware selection to jointly optimize accuracy, efficiency, and stability. Experiments on two real-world psychological assessment datasets demonstrate that OAT improves average classification accuracy by 12.7% over baseline methods, while offering strong interpretability and practical utility.

Technology Category

• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Search and Optimization: Evolutionary Computation
• Machine Learning: Evolutionary Learning

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

With the rapid advancement of intelligent education, Computerized Adaptive Testing (CAT) has attracted increasing attention by integrating educational psychology with deep learning technologies. Unlike traditional paper-and-pencil testing, CAT aims to efficiently and accurately assess examinee abilities by adaptively selecting the most suitable items during the assessment process. However, its real-time and sequential nature presents limitations in practical scenarios, particularly in large-scale assessments where interaction costs are high, or in sensitive domains such as psychological evaluations where minimizing noise and interference is essential. These challenges constrain the applicability of conventional CAT methods in time-sensitive or resourceconstrained environments. To this end, we first introduce a novel task called one-shot adaptive testing (OAT), which aims to select a fixed set of optimal items for each test-taker in a one-time selection. Meanwhile, we propose PEOAT, a Personalization-guided Evolutionary question assembly framework for One-shot Adaptive Testing from the perspective of combinatorial optimization. Specifically, we began by designing a personalization-aware initialization strategy that integrates differences between examinee ability and exercise difficulty, using multi-strategy sampling to construct a diverse and informative initial population. Building on this, we proposed a cognitive-enhanced evolutionary framework incorporating schema-preserving crossover and cognitively guided mutation to enable efficient exploration through informative signals. To maintain diversity without compromising fitness, we further introduced a diversity-aware environmental selection mechanism. The effectiveness of PEOAT is validated through extensive experiments on two datasets, complemented by case studies that uncovered valuable insights.

Problem

• Develops one-shot adaptive testing for efficient large-scale assessments
• Addresses high interaction costs in real-time computerized adaptive testing
• Minimizes noise in sensitive domains like psychological evaluations

Innovation

• Personalization-aware initialization strategy integrating ability-differences
• Cognitive-enhanced evolutionary framework with schema-preserving crossover
• Diversity-aware environmental selection mechanism maintaining fitness balance