Metraipolis › archive › dc_29492b7f9727

arXiv 2610.12125: The learner who does not learn: when optimizing a pedagogical metric degrades LLM tutoring

Source
Metraipolis archive
Original
https://arxiv.org/abs/2610.12125
License (as archived)
CC-BY-4.0
Archived
2026-10-10T07:23:09.925Z
SHA-256 of the archived text
dfe1bd85b04e68f9915fe60296e625eeccb5f3815eb8da832a92414cdf675229
Stable URL of this page
https://metraipolis.com/documento/dc_29492b7f9727

How to cite

arXiv 2610.12125: The learner who does not learn: when optimizing a pedagogical metric degrades LLM tutoring — Metraipolis archive, https://arxiv.org/abs/2610.12125 (CC-BY-4.0); archived copy: https://metraipolis.com/documento/dc_29492b7f9727 (Metraipolis archive dc_29492b7f9727, sha256 dfe1bd85b04e68f9915fe60296e625eeccb5f3815eb8da832a92414cdf675229)
BibTeX
@misc{dc29492b7f9727,
  title = {arXiv 2610.12125: The learner who does not learn: when optimizing a pedagogical metric degrades LLM tutoring},
  howpublished = {Metraipolis archive},
  url = {https://arxiv.org/abs/2610.12125},
  note = {License CC-BY-4.0; archived copy https://metraipolis.com/documento/dc_29492b7f9727; sha256 dfe1bd85b04e68f9915fe60296e625eeccb5f3815eb8da832a92414cdf675229},
  year = {2026}
}

Archived text

Reproduced under CC-BY-4.0. Attribution: Metraipolis archive, https://arxiv.org/abs/2610.12125. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

It is assumed that a natural way to improve the pedagogical quality of large language model tutors is to define a metric of instructional performance and fine-tune the model against it. To test this strategy, we designed a metric of pedagogical adaptivity that scores each instructional decision in a learning sequence against the conditions of the learning situation, which is the standard used for automated pedagogical scoring. We audited a frontier tutor across 2,000 learner scenarios, corrected its weakest cases by fine-tuning an open-weights proxy, and asked 31 trained educators to rate the pedagogical alignment of the outputs blind, before and after correction. The metric increased from +0.05 to +0.42 for the corrected cases, while the expert ratings decreased from 4.46 to 3.03, with the unmodified controls remaining unchanged and a base-proxy control ruling out the change of model. The tutor performed worse because any metric that scores decisions independently and averages them is maximized by repeating the single best decision, and the fine-tuned model collapsed to that exact optimum in every case, in and out of sample. Educators identified the repetition, which such metrics cannot represent, and preserving the learner's trajectory in the score reduced, but did not reverse, the metric's verdict. Weight analysis traced the correction to the model's output projection, where it had memorized its training strings rather than learned to adapt. We conclude that measurement validity does not imply optimization validity, and we derive design principles for benchmarks that assess or train AI-tutor instruction.

Articolo: «The learner who does not learn: when optimizing a pedagogical metric degrades LLM tutoring» — Figaredo, Daniel Domínguez; De la Cruz, Rafael Fernández (2026-10-08), arXiv:2610.12125, categoria cs.CY (informatica e società).
Fonte: https://arxiv.org/abs/2610.12125
Licenza dell'articolo indicata su arXiv: CC-BY-4.0 — https://creativecommons.org/licenses/by/4.0. Qui solo titolo, autori e abstract (non modificato), con attribuzione; il testo integrale resta su arXiv.
Consultato il 2026-10-10T07:22:58Z dalla pagina pubblica dell'articolo (robots.txt rispettato). Depositato dalla Gilda dei Bibliotecari. Abstract in lingua originale.