Metraipolis › archive › dc_6dc07fcf46b3

Vision-Language Models for Criterion-Level Grading of Handwritten Examinations in Outcome-Based Education — arXiv:2609.14284

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.14284
License (as archived)
CC0-1.0
Archived
2026-10-16T01:52:54.946Z
SHA-256 of the archived text
1383c146c30c72de7d74630cba3ee29388c5b8c634a23925966aad99bab89741
Stable URL of this page
https://metraipolis.com/documento/dc_6dc07fcf46b3

How to cite

Vision-Language Models for Criterion-Level Grading of Handwritten Examinations in Outcome-Based Education — arXiv:2609.14284 — arXiv (en), https://arxiv.org/abs/2609.14284 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6dc07fcf46b3 (Metraipolis archive dc_6dc07fcf46b3, sha256 1383c146c30c72de7d74630cba3ee29388c5b8c634a23925966aad99bab89741)
BibTeX
@misc{dc6dc07fcf46b3,
  title = {Vision-Language Models for Criterion-Level Grading of Handwritten Examinations in Outcome-Based Education — arXiv:2609.14284},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.14284},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6dc07fcf46b3; sha256 1383c146c30c72de7d74630cba3ee29388c5b8c634a23925966aad99bab89741},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.14284. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Criterion-level grading connects examination performance to learning outcomes, but manual marking introduces workload and variation between markers. This study evaluates vision-language models (VLMs) for handwritten outcome-based assessment across five dimensions: accuracy, human agreement, repeated-run reliability, error concentration, and explanation quality. Using 1,982 criterion-level records from 485 undergraduate examination answers, we compare 20 configurations spanning Qwen2.5-VL, InternVL3, Pixtral, a Donut baseline, and a cascade ensemble. Evaluation setups include zero-shot prompting, few-shot prompting, partial fine-tuning, and Low-Rank Adaptation (LoRA). Two independent faculty markers regraded all 291 test criteria, providing a human agreement baseline on the same assessment materials. Qwen2.5-VL with LoRA achieved Quadratic Weighted Kappa (QWK) of 0.727 and mean absolute error of 0.435 marks against the examiner, compared with mean human-pair QWK of 0.551. This comparison reflects calibration to the examiner's training marks. LoRA outperformed partial fine-tuning for all three instruction-tuned VLMs, while few-shot prompting reduced QWK in every configuration with valid prompted scores. Aggregate reliability and exact repeatability diverged: intraclass correlations ranged from 0.790 to 0.874, yet 50.2-63.6% of criteria changed marks across five sampled runs. Attention-guided deletion showed no statistically significant advantage over random masking, and four faculty reviewers reached no consensus on explanation usefulness. These findings highlight the need for rubric-specific calibration, repeatable scoring, review of consequential errors, and separate validation of explanations. The released evaluation protocol supports criterion-level assessment research and grading tools with teacher oversight.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-15 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2609.14284 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T01:25:23.566Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-15?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).