Metraipolis › archive › dc_be6ff3aa1c88
Large Scale AI Grading of Handwritten Physics Assessments: Score Agreement and Olympiad Team Selection Outcomes — arXiv:2608.20521
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.20521
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T11:09:12.291Z
- SHA-256 of the archived text
b0f335a19721d2a6c6c4ead387c076ad36ed9072c07309a215775f182f2f098f- Stable URL of this page
https://metraipolis.com/documento/dc_be6ff3aa1c88
How to cite
Large Scale AI Grading of Handwritten Physics Assessments: Score Agreement and Olympiad Team Selection Outcomes — arXiv:2608.20521 — arXiv (en), https://arxiv.org/abs/2608.20521 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_be6ff3aa1c88 (Metraipolis archive dc_be6ff3aa1c88, sha256 b0f335a19721d2a6c6c4ead387c076ad36ed9072c07309a215775f182f2f098f)
BibTeX
@misc{dcbe6ff3aa1c88,
title = {Large Scale AI Grading of Handwritten Physics Assessments: Score Agreement and Olympiad Team Selection Outcomes — arXiv:2608.20521},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.20521},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_be6ff3aa1c88; sha256 b0f335a19721d2a6c6c4ead387c076ad36ed9072c07309a215775f182f2f098f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.20521. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Multimodal AI can read handwritten physics solutions, but high-stakes grading requires agreement with official scores and outcomes. This study evaluated GPT-5.5-based grading on 10364 scanned pages from 520 handwritten submissions by 416 unique candidates or students across three assessments: a national Physics Olympiad theory examination, the final Olympiad selection camp with theory and experiment components, and a university quantum-mechanics examination. Each submission was graded twice by AI using the official rubrics. The second round used revised page-by-page and evidence-location instructions developed after first-round disagreement analysis. During grading, AI did not see official human marks or AI--human comparisons. Total-score correlations with official marks were high (0.91--0.97). For the final Olympiad selection, AI recovered the same five-student team as official grading. The second round improved aggregate question-part agreement, especially where first-round disagreements were larger. The main difficulty remained exact partial-credit grading, especially in experimental work. Reliable AI grading therefore depends on detailed rubrics and should be used as a second reader or audit tool under examiner control. Tema degli esploratori: ricerca e scienza aperta (retro-catalogo arXiv: annunci del 2026-08-24 in physics.ed-ph). Categorie arXiv: Physics Education (physics.ed-ph) ; Artificial Intelligence (cs.AI); Computers and Society (cs.CY). Pagina del lavoro: https://arxiv.org/abs/2608.20521 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-18T11:08:14.656Z dalla pagina https://arxiv.org/catchup/physics.ed-ph/2026-08-24?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).