Metraipolis › archive › dc_bf142d229ecc

Signed Rescue Routing: Harm-Aware Cascades for Efficient LLM Inference — arXiv:2609.07786

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.07786
License (as archived): CC0-1.0
Archived: 2026-10-16T12:45:46.475Z
SHA-256 of the archived text: b6ff981a2caac9e65adc72b923d793d6534746ccf87b8218fe1681a4143eae97
Stable URL of this page: https://metraipolis.com/documento/dc_bf142d229ecc

How to cite

Signed Rescue Routing: Harm-Aware Cascades for Efficient LLM Inference — arXiv:2609.07786 — arXiv (en), https://arxiv.org/abs/2609.07786 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bf142d229ecc (Metraipolis archive dc_bf142d229ecc, sha256 b6ff981a2caac9e65adc72b923d793d6534746ccf87b8218fe1681a4143eae97)

BibTeX

@misc{dcbf142d229ecc,
  title = {Signed Rescue Routing: Harm-Aware Cascades for Efficient LLM Inference — arXiv:2609.07786},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.07786},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bf142d229ecc; sha256 b6ff981a2caac9e65adc72b923d793d6534746ccf87b8218fe1681a4143eae97},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.07786. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language model (LLM) cascades answer easy requests with a small model and escalate selected requests to a larger model. Most routers prioritize examples on which the small model appears uncertain or likely to be wrong. This proxy ignores a decisive fact: escalation is useful only when the large model corrects the small model, and it is harmful when the large model replaces a correct answer with an incorrect one. We introduce Signed Rescue Routing (SRR), a budgeted routing method that predicts these two events separately and ranks requests by their difference. We show that this signed conditional gain is the Bayes-optimal routing score under a fixed escalation budget. SRR requires only the small model's output statistics at deployment and adds a lightweight two-head router. We evaluate SRR with Qwen3-4B and Qwen3-8B on TBD examples from MMLU, HellaSwag, and ARC-Challenge. Across the accuracy-compute curve, SRR reaches an area of TBD, compared with TBD for a learned small-model error predictor and TBD for entropy routing. These results show that predicting incremental value, rather than model uncertainty, is a simple and effective objective for efficient LLM cascades.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-09 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.07786 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T12:30:25.142Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-09?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).