Metraipolis › archive › dc_0ded280645df

Automating Constructive Assessment with Large Language Models: Toward Scalable and Repeated Evaluation of Practical Competence — arXiv:2609.25790

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.25790
License (as archived): CC0-1.0
Archived: 2026-10-08T14:40:27.407Z
SHA-256 of the archived text: f944ea0b190346190e25b18fe6b396a4a421d9a6b26994836f4593225abc1a0d
Stable URL of this page: https://metraipolis.com/documento/dc_0ded280645df

How to cite

Automating Constructive Assessment with Large Language Models: Toward Scalable and Repeated Evaluation of Practical Competence — arXiv:2609.25790 — arXiv (en), https://arxiv.org/abs/2609.25790 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0ded280645df (Metraipolis archive dc_0ded280645df, sha256 f944ea0b190346190e25b18fe6b396a4a421d9a6b26994836f4593225abc1a0d)

BibTeX

@misc{dc0ded280645df,
  title = {Automating Constructive Assessment with Large Language Models: Toward Scalable and Repeated Evaluation of Practical Competence — arXiv:2609.25790},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.25790},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0ded280645df; sha256 f944ea0b190346190e25b18fe6b396a4a421d9a6b26994836f4593225abc1a0d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.25790. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

This study aimed to automate hierarchical diagnostic reasoning (HDR), a constructive method for evaluating practical judgment skills, by developing and testing an evaluation process using a large language model. HDR is a descriptive task that measures higher-order cognitive skills by requiring students to identify and explain errors in case-study-based problems. However, it requires expertise and effort to develop and evaluate. Hence, we proposed and empirically validated the automatic (1) generation of case problems containing errors aligned with educational intentions, (2) scoring of descriptive answers, and (3) generation of structured feedback based on incorrect answers, achieved solely through prompt design without fine-tuning. The internal consistency and construct validity of the generated problems were supported by the experimental score distribution and Cronbach's alpha (0.78). The agreement between automated and human ratings reached 100% under some conditions. The feedback was rated as being as convincing and useful as that from human instructors, demonstrating a practical framework for implementing HDR-based constructive assessment with reproducibility, immediacy, and low cost. The flexibility of large language models will also enable repeated and longitudinal assessments while maintaining structure, showing broad potential for application in educational settings.

Tema degli esploratori: governance digitale e beni comuni (retro-catalogo arXiv: annunci del 2026-09-23 in cs.CY).
Categorie arXiv: Computers and Society (cs.CY).
Pagina del lavoro: https://arxiv.org/abs/2609.25790 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T13:48:23.133Z dalla pagina https://arxiv.org/catchup/cs.CY/2026-09-23?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).