Metraipolis › archive › dc_007dd7dfcde2

Limited Structural Reliability in Public Educational Prediction Benchmarks: A Four-Dimension Audit of Seven Datasets — arXiv:2609.29625

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.29625
License (as archived)
CC0-1.0
Archived
2026-10-08T05:26:07.077Z
SHA-256 of the archived text
c04033434ed34b10dcc0f00d882bd856222718ada1f9d5c631a629270e14c0e7
Stable URL of this page
https://metraipolis.com/documento/dc_007dd7dfcde2

How to cite

Limited Structural Reliability in Public Educational Prediction Benchmarks: A Four-Dimension Audit of Seven Datasets — arXiv:2609.29625 — arXiv (en), https://arxiv.org/abs/2609.29625 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_007dd7dfcde2 (Metraipolis archive dc_007dd7dfcde2, sha256 c04033434ed34b10dcc0f00d882bd856222718ada1f9d5c631a629270e14c0e7)
BibTeX
@misc{dc007dd7dfcde2,
  title = {Limited Structural Reliability in Public Educational Prediction Benchmarks: A Four-Dimension Audit of Seven Datasets — arXiv:2609.29625},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.29625},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_007dd7dfcde2; sha256 c04033434ed34b10dcc0f00d882bd856222718ada1f9d5c631a629270e14c0e7},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.29625. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Across seven public educational prediction datasets, three passed all four pre-modeling reliability checks; the remaining four either failed group-aware generalization tests or lacked the provenance metadata needed to run them. One dataset was initially classified as failing but corrected after excluding group-identifier features from the holdout matrix, demonstrating that the audit can distinguish genuine cross-group confounding from feature-encoding artifacts. Each dataset was audited before model optimization using four checks: baseline gap, split instability, null separation, and metadata adequacy under group-aware holdout. The dominant failure mode was not weak iid performance alone but cross-group fragility: in the clearest case, UCI Student declined from iid R-squared 0.242 to group-holdout R-squared -0.097, while Higher Ed collapsed from 0.041 to -8.79. Increasing model complexity did not remove this pattern: ensemble models improved structurally sound datasets but amplified instability or failed under group holdout on fragile ones. An exploratory cross-dataset comparison further showed that stronger profiles clustered in larger, richer-grouped, performance-proximal datasets, while random-split performance severely overstated deployable signal in fragile datasets. Classification-metric sensitivity analyses reached the same substantive conclusions. The results show that benchmark reliability in educational AI is constrained less by algorithm choice than by data structure, group heterogeneity, and evaluation design. A reusable pre-modeling audit offers a minimum quality gate before public educational datasets support strong benchmark or deployment claims.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-25 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.29625 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T04:50:39.452Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).