Metraipolis › archive › dc_51111debd7ec

Measuring the Partial-Credit Gap: A Strict Benchmark on Vietnam's 2025 Convex Marking Scheme — arXiv:2608.18336

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.18336
License (as archived)
CC0-1.0
Archived
2026-10-18T13:44:28.072Z
SHA-256 of the archived text
ec6e66b647db804830e3d304f2b70f536dd9f9c3b17096cedcf8767906670da3
Stable URL of this page
https://metraipolis.com/documento/dc_51111debd7ec

How to cite

Measuring the Partial-Credit Gap: A Strict Benchmark on Vietnam's 2025 Convex Marking Scheme — arXiv:2608.18336 — arXiv (en), https://arxiv.org/abs/2608.18336 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_51111debd7ec (Metraipolis archive dc_51111debd7ec, sha256 ec6e66b647db804830e3d304f2b70f536dd9f9c3b17096cedcf8767906670da3)
BibTeX
@misc{dc51111debd7ec,
  title = {Measuring the Partial-Credit Gap: A Strict Benchmark on Vietnam's 2025 Convex Marking Scheme — arXiv:2608.18336},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.18336},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_51111debd7ec; sha256 ec6e66b647db804830e3d304f2b70f536dd9f9c3b17096cedcf8767906670da3},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.18336. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

When evaluating language models on human exams, benchmarks typically score each response as right or wrong and report the overall accuracy. This approach assumes that partial knowledge is worth proportional credit, an assumption that fails when an examination uses a non-additive grading scheme. The 2025 reform of Vietnam's National High School Graduation Examination demonstrates the cost of this substitution. In Part II of the exam, candidates evaluate four true/false statements per question. The grading is convex: the number of correct statements earns 0, 0.10, 0.25, 0.50, or 1.00 points. Identifying three statements correctly pays 0.50 points, not the 0.75 points that standard accuracy metrics would award. Because Part II accounts for 4.00 of the exam's 10.00 points, reporting accuracy inflates the score by rewarding partial knowledge that the state explicitly penalizes. We introduce THPT-Ladder, a benchmark of 632 items from 21 official exams across 11 subjects, graded exactly as the ministry grades its students. The ministry publishes the marks of over a million candidates, allowing us to place models directly into the human cohort. Across eight models, the official rubric pays 0.020 to 0.159 points less per Part II question than proportional credit. This shortfall changes a model's apparent competence. For Qwen3.5-27B on the 2025 History exam, a 0.042-point shortfall drops its standing from the 90th to the 77th percentile among 481,293 candidates. A model's accuracy does not predict this penalty. At Claude Sonnet 5's accuracy level, different distributions of errors yield scores varying from 0.869 to 0.932 points per question. Official marks depend on how correct statements are grouped, meaning standard benchmarks report a competence the institution would not certify.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-20 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computers and Society (cs.CY).
Pagina del lavoro: https://arxiv.org/abs/2608.18336 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T13:41:17.859Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-20?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).