Metraipolis › archive › dc_4e36103ae6c7
Accurate Ensembles, Fragile Narratives: Multi-Scale Stacking and a Fidelity Audit of LLM-Generated Explanations for Credit Risk — arXiv:2608.08126
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.08126
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T08:50:43.098Z
- SHA-256 of the archived text
8a1354543f8173859a4b406740b2578ca27b4e0a283565a5caab4fd9092aef68- Stable URL of this page
https://metraipolis.com/documento/dc_4e36103ae6c7
How to cite
Accurate Ensembles, Fragile Narratives: Multi-Scale Stacking and a Fidelity Audit of LLM-Generated Explanations for Credit Risk — arXiv:2608.08126 — arXiv (en), https://arxiv.org/abs/2608.08126 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4e36103ae6c7 (Metraipolis archive dc_4e36103ae6c7, sha256 8a1354543f8173859a4b406740b2578ca27b4e0a283565a5caab4fd9092aef68)
BibTeX
@misc{dc4e36103ae6c7,
title = {Accurate Ensembles, Fragile Narratives: Multi-Scale Stacking and a Fidelity Audit of LLM-Generated Explanations for Credit Risk — arXiv:2608.08126},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.08126},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4e36103ae6c7; sha256 8a1354543f8173859a4b406740b2578ca27b4e0a283565a5caab4fd9092aef68},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.08126. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Credit scoring increasingly relies on models whose decision logic cannot be read off their parameters, in tension with supervisory expectations that adverse decisions be explainable. A common proposal closes that gap with a language model: compute feature attributions, hand them to an LLM, and let it write the rationale. We build such a system end to end and test whether the second half of the promise holds. The predictive component is a multi-scale stacking ensemble fusing four differently regularised gradient-boosting learners with a residual network through a neural meta-learner trained on out-of-fold predictions. On a public 32,581-application credit dataset it reaches test ROC-AUC 0.9539 (95% CI [0.9462, 0.9616]) and PR-AUC 0.9137, beating the best single model by Delta-AUC = 0.0143 (p = 0.016 under a conservative independence assumption). Our central finding is asymmetric. The ranking gain is real but operationally small: at the F1-optimal threshold the ensemble avoids only six additional missed defaults out of 1,422 against a tuned random forest, cutting cost-weighted loss by under 2%. The narrative layer fails in a way prompt engineering alone does not fix. In an audited case the model named three factors as risk-increasing that the supplied attributions scored as risk-reducing, omitted the dominant driver, and introduced a feature never given to it. We trace this to properties we measure rather than assume: SHAP and LIME agree on which features matter (overlap@10 = 0.80) but not on their order (tau = 0.43, p = 0.18), and the attribution sign for the model's most sensitive input is near a coin flip across applicants (modal-sign share 0.53). Calibration (ECS = 0.117) and perturbation stability (DPD = 0.078) both fall short of our own thresholds. Constrained prompting is necessary but not sufficient: grounding must be verified after generation, not assumed. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-11 in cs.CL). Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2608.08126 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T08:49:41.972Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).