Large Scale AI Grading of Handwritten Physics Assessments: Score Agreement and Olympiad Team Selection Outcomes

2026-08-20

AI Summary

研究使用GPT-5.5评估了10364页手写物理答案的评分，通过两轮改进后与官方评分高度一致，适用于辅助高风险考试评分。

Abstract

Multimodal AI can read handwritten physics solutions, but high-stakes grading requires agreement with official scores and outcomes. This study evaluated GPT-5.5-based grading on 10364 scanned pages from 520 handwritten submissions by 416 unique candidates or students across three assessments: a national Physics Olympiad theory examination, the final Olympiad selection camp with theory and experiment components, and a university quantum-mechanics examination. Each submission was graded twice by AI using the official rubrics. The second round used revised page-by-page and evidence-location instructions developed after first-round disagreement analysis. During grading, AI did not see official human marks or AI--human comparisons. Total-score correlations with official marks were high (0.91--0.97). For the final Olympiad selection, AI recovered the same five-student team as official grading. The second round improved aggregate question-part agreement, especially where first-round disagreements were larger. The main difficulty remained exact partial-credit grading, especially in experimental work. Reliable AI grading therefore depends on detailed rubrics and should be used as a second reader or audit tool under examiner control.

Problem

• AI Grading
• Handwritten Physics Solutions
• Score Agreement
• Olympiad Team Selection
• Partial-Credit Grading

Innovation

• Multimodal AI
• GPT-5.5-based grading
• handwritten physics solutions
• detailed rubrics
• second reader or audit tool