Metraipolis › archive › dc_c2ca1b47f66b

SHIFT-JEV: Certified Abstention and Gain-Gated Continual Adaptation for LLM-Based Decision Models under Distribution Shift — arXiv:2610.34329

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.34329
License (as archived)
CC0-1.0
Archived
2026-10-27T08:01:19.363Z
SHA-256 of the archived text
d64dff68f31b8b6f88a19ab6fbeadd2fdf8ea0ac3977fb367e759f0bbb39b9f1
Stable URL of this page
https://metraipolis.com/documento/dc_c2ca1b47f66b

How to cite

SHIFT-JEV: Certified Abstention and Gain-Gated Continual Adaptation for LLM-Based Decision Models under Distribution Shift — arXiv:2610.34329 — arXiv (en), https://arxiv.org/abs/2610.34329 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c2ca1b47f66b (Metraipolis archive dc_c2ca1b47f66b, sha256 d64dff68f31b8b6f88a19ab6fbeadd2fdf8ea0ac3977fb367e759f0bbb39b9f1)
BibTeX
@misc{dcc2ca1b47f66b,
  title = {SHIFT-JEV: Certified Abstention and Gain-Gated Continual Adaptation for LLM-Based Decision Models under Distribution Shift — arXiv:2610.34329},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.34329},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c2ca1b47f66b; sha256 d64dff68f31b8b6f88a19ab6fbeadd2fdf8ea0ac3977fb367e759f0bbb39b9f1},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.34329. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) can act as decision models that return a probability distribution over a fixed set of options, but deployed decision models face domain, label-prior, emerging-class and candidate-set shift. We study how such models fail under shift and propose SHIFT-Jev, a monitoring, abstention and continual-adaptation procedure for them. It scores the risk of each decision from uncertainty, format and option-consistency signals, certifies abstention thresholds with Learn-then-Test, monitors the decision stream with drift and label-shift detectors, and accepts a model update only when an expected-gain gate finds a significant held-out improvement without loss on earlier regimes. On a benchmark of ten shifted test environments with Qwen3-0.6B and Qwen3-4B (three seeds each) and Phi-4-mini, fine-tuning with KL anchors and temperature scaling lowers calibration error under shift from 0.289 to 0.087. At a target risk of 10%, a learned uncertainty score with a certified threshold exceeds the target in 3 to 5 of 27 environment-seed tests, against 5 to 15 for a max-probability score with the same certification and 9 to 14 for conformal prediction sets; drift-based routing changes little. A chi-squared test on predicted labels detects 95 to 99% of class-prior shifts that confidence- and embedding-based detectors miss. In continual adaptation, SHIFT-Jev reaches 70.5% accuracy on Qwen3-0.6B, above replay, LwF, EWC and fine-tuning baselines in every seed, whereas on Qwen3-4B its conservative gate leaves it 0.4 to 1.4 points behind them.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.34329 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-27T07:55:30.892Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).