Metraipolis › archive › dc_0bee26784192

Beyond Response Scoring: An AI-aware Cognitive Supervision Framework for Automated Mathematical Short Answer Grading — arXiv:2610.41215

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.41215
License (as archived): CC0-1.0
Archived: 2026-11-02T07:33:23.758Z
SHA-256 of the archived text: 6c5b84b86e0cfc55ada90c7d40d1d150fa59deead0879789ef6f406b02e73565
Stable URL of this page: https://metraipolis.com/documento/dc_0bee26784192

How to cite

Beyond Response Scoring: An AI-aware Cognitive Supervision Framework for Automated Mathematical Short Answer Grading — arXiv:2610.41215 — arXiv (en), https://arxiv.org/abs/2610.41215 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0bee26784192 (Metraipolis archive dc_0bee26784192, sha256 6c5b84b86e0cfc55ada90c7d40d1d150fa59deead0879789ef6f406b02e73565)

BibTeX

@misc{dc0bee26784192,
  title = {Beyond Response Scoring: An AI-aware Cognitive Supervision Framework for Automated Mathematical Short Answer Grading — arXiv:2610.41215},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.41215},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0bee26784192; sha256 6c5b84b86e0cfc55ada90c7d40d1d150fa59deead0879789ef6f406b02e73565},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41215. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Transformer-based language models have achieved remarkable success in automated short answer grading (ASAG). However, current methods in ASAG struggle to comprehend, reason about and evaluate complex mathematical answers involving symbols and formulas. These score-only methods fail to generalize across diverse responses and skill states associated with identical scores. Moreover, they ignore the potential upward bias introduced by generative Artificial Intelligence (AI) usage in assessing students' performance and knowledge mastery. To address these limitations, we propose AICG (AI-aware Cognitive-supervised Grading), a two-branch multi-task framework. The first branch combines RoBERTa-based regression, ordinal, and classification heads for individual-answer grading. The second branch uses an expert-designed Q-matrix to associate responses with assessed skills and jointly predicts G-DINA-derived skill mastery and learning gain. Through the shared encoder, these skill-level targets provide cognitive supervision for answer grading beyond score labels. AICG further uses a Gen-AI detector, trained through supervised contrastive pretraining followed by linear classification, to estimate assistance probabilities and convert them into reliability weights, attenuating unreliable cognitive evidence during learning assessment. We conduct experiments on 49,402 instructor-graded responses spanning 19 academic semesters. Relative to RoBERTa regression, AICG reduces mean absolute error (MAE) by 50.3% and improves quadratic weighted kappa (QWK) from 0.798 to 0.927. Ablations identify Q-matrix-guided cognitive supervision as the primary source of improvement, while AI-aware weighting further reduces overall MAE by 4.2% over the unweighted cognitive model and consistently improves MAE and QWK across the four most recent post-Gen-AI semesters.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.41215 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-02T07:27:12.627Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).