🤖 AI Summary
This work addresses the challenge of automatic essay scoring, which requires predicting absolute scores—a task difficult to model directly. To overcome this, the authors propose Pair2Score, a two-stage framework that first trains a directional siamese ranker to perform pairwise comparisons and then efficiently transfers the learned ranking knowledge to an absolute scoring model via configurable transfer strategies, such as warm-start initialization and embedding fusion. This approach innovatively integrates pairwise comparison with absolute scoring and demonstrates that the choice of transfer configuration critically influences performance. Experimental results across grammatical, lexical, and syntactic tasks show that the best-performing transfer variants consistently and significantly outperform pure absolute-scoring baselines, achieving notable gains in Quadratic Weighted Kappa (QWK) while enabling parameter-efficient fine-tuning based on LLaMA.
📝 Abstract
Many scoring applications require absolute predictions, while pairwise comparisons can provide a simpler learning objective. We present Pair2Score, a two-stage learning framework that transfers pairwise comparisons into absolute scoring with parameter-efficient LLaMA adaptation. Stage 1 trains a directional Siamese ranker on pairwise comparisons derived from absolute trait labels; Stage 2 trains an absolute predictor using configurable transfer strategies (warm-start and embedding-fusion variants). We evaluate on rubric-aligned Automated Essay Scoring (AES) traits (grammar, vocabulary, syntax) under a five-fold protocol that co-rotates held-out fold and random seed. At the trait level, the best-performing transfer variant improves quadratic weighted kappa (QWK) over an absolute-only baseline for all three traits. However, not all transfer configurations help: a one-epoch pairwise stage transfers more reliably than extended pairwise training, and transfer configuration -- not just the inclusion of a pairwise stage -- determines whether downstream scoring benefits.