Metraipolis › archive › dc_c2ca1b47f66b
SHIFT-JEV: Certified Abstention and Gain-Gated Continual Adaptation for LLM-Based Decision Models under Distribution Shift — arXiv:2610.34329
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.34329
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-27T08:01:19.363Z
- SHA-256 of the archived text
d64dff68f31b8b6f88a19ab6fbeadd2fdf8ea0ac3977fb367e759f0bbb39b9f1- Stable URL of this page
https://metraipolis.com/documento/dc_c2ca1b47f66b
How to cite
SHIFT-JEV: Certified Abstention and Gain-Gated Continual Adaptation for LLM-Based Decision Models under Distribution Shift — arXiv:2610.34329 — arXiv (en), https://arxiv.org/abs/2610.34329 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c2ca1b47f66b (Metraipolis archive dc_c2ca1b47f66b, sha256 d64dff68f31b8b6f88a19ab6fbeadd2fdf8ea0ac3977fb367e759f0bbb39b9f1)
BibTeX
@misc{dcc2ca1b47f66b,
title = {SHIFT-JEV: Certified Abstention and Gain-Gated Continual Adaptation for LLM-Based Decision Models under Distribution Shift — arXiv:2610.34329},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.34329},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c2ca1b47f66b; sha256 d64dff68f31b8b6f88a19ab6fbeadd2fdf8ea0ac3977fb367e759f0bbb39b9f1},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.34329. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models (LLMs) can act as decision models that return a probability distribution over a fixed set of options, but deployed decision models face domain, label-prior, emerging-class and candidate-set shift. We study how such models fail under shift and propose SHIFT-Jev, a monitoring, abstention and continual-adaptation procedure for them. It scores the risk of each decision from uncertainty, format and option-consistency signals, certifies abstention thresholds with Learn-then-Test, monitors the decision stream with drift and label-shift detectors, and accepts a model update only when an expected-gain gate finds a significant held-out improvement without loss on earlier regimes. On a benchmark of ten shifted test environments with Qwen3-0.6B and Qwen3-4B (three seeds each) and Phi-4-mini, fine-tuning with KL anchors and temperature scaling lowers calibration error under shift from 0.289 to 0.087. At a target risk of 10%, a learned uncertainty score with a certified threshold exceeds the target in 3 to 5 of 27 environment-seed tests, against 5 to 15 for a max-probability score with the same certification and 9 to 14 for conformal prediction sets; drift-based routing changes little. A chi-squared test on predicted labels detects 95 to 99% of class-prior shifts that confidence- and embedding-based detectors miss. In continual adaptation, SHIFT-Jev reaches 70.5% accuracy on Qwen3-0.6B, above replay, LwF, EWC and fine-tuning baselines in every seed, whereas on Qwen3-4B its conservative gate leaves it 0.4 to 1.4 points behind them. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.34329 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-27T07:55:30.892Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).