Metraipolis › archive › dc_a1716bd6d2cf

Distilling Black-Box Machine Learning into a Small, Self-Explaining Language Model for Learning Analytics — arXiv:2608.21165

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.21165
License (as archived)
CC0-1.0
Archived
2026-10-18T10:27:30.617Z
SHA-256 of the archived text
9ccf258d02fe8574172ecd6ef1241e7ff3a8affc74a053c7e9662ee3c9db27f7
Stable URL of this page
https://metraipolis.com/documento/dc_a1716bd6d2cf

How to cite

Distilling Black-Box Machine Learning into a Small, Self-Explaining Language Model for Learning Analytics — arXiv:2608.21165 — arXiv (en), https://arxiv.org/abs/2608.21165 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a1716bd6d2cf (Metraipolis archive dc_a1716bd6d2cf, sha256 9ccf258d02fe8574172ecd6ef1241e7ff3a8affc74a053c7e9662ee3c9db27f7)
BibTeX
@misc{dca1716bd6d2cf,
  title = {Distilling Black-Box Machine Learning into a Small, Self-Explaining Language Model for Learning Analytics — arXiv:2608.21165},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.21165},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a1716bd6d2cf; sha256 9ccf258d02fe8574172ecd6ef1241e7ff3a8affc74a053c7e9662ee3c9db27f7},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.21165. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Learning analytics increasingly relies on flexible machine learning (ML), but the model opacity and the burden of deployment prevent these tools from reaching educational practice. We propose a two-stage fine-tuning pipeline that distills a fitted black-box estimator and its post hoc interpretation (the mentor) into a small, open-weight large language model (LLM; the mentee) that returns an individual-level estimate and explains in natural language. The design is estimator-agnostic and paired with a faithfulness-first evaluation framework that audits every narration against the attribution it claims to describe. We design a simulation study that separates distillation loss from estimator loss by comparing an oracle mentor with a realistic ML mentor. Given an oracle signal, distillation with a two-billion-parameter LLM model is nearly lossless in recovering the effect surface (r > .90), perfectly ranking the important variables, and citing no spurious covariate. Under a realistic estimator, almost all remaining error originates upstream. We find that fluency is no evidence of correctness since narration quality is independent of signal quality, and decision quality collapses toward the majority action in severely imbalanced settings. Applied to a nationally representative dataset, the pipeline recovers the finding that advanced mathematics coursework benefits students least likely to enroll in four-year college the most, with 98.8% of narrations passing the audit and no fabricated quantities. The result is a single fine-tuned LLM that predicts and explains offline on a commodity laptop, so student records never leave the machine.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-24 in cs.HC).
Categorie arXiv: Human-Computer Interaction (cs.HC) ; Computers and Society (cs.CY).
Pagina del lavoro: https://arxiv.org/abs/2608.21165 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T10:27:20.506Z dalla pagina https://arxiv.org/catchup/cs.HC/2026-08-24?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).