Metraipolis › archive › dc_8edd9875ba26
InferOpt: Constrained Multi-Objective Search for LLM Inference Configurations — arXiv:2610.04473
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.04473
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-06T05:24:35.955Z
- SHA-256 of the archived text
3c1c44487c2174de51440fcdee5209996864f6ec4a54576a6a992785c52ebd32- Stable URL of this page
https://metraipolis.com/documento/dc_8edd9875ba26
How to cite
InferOpt: Constrained Multi-Objective Search for LLM Inference Configurations — arXiv:2610.04473 — arXiv, https://arxiv.org/abs/2610.04473 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8edd9875ba26 (Metraipolis archive dc_8edd9875ba26, sha256 3c1c44487c2174de51440fcdee5209996864f6ec4a54576a6a992785c52ebd32)
BibTeX
@misc{dc8edd9875ba26,
title = {InferOpt: Constrained Multi-Objective Search for LLM Inference Configurations — arXiv:2610.04473},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.04473},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8edd9875ba26; sha256 3c1c44487c2174de51440fcdee5209996864f6ec4a54576a6a992785c52ebd32},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.04473. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Serving an LLM means setting dozens of inference-time knobs, from per-layer KV retention to per-layer expert counts. Practice sets them with mechanism-specific heuristics that return a single operating point and do not scale to layer-wise search spaces. We recast inference configuration as constrained multi-objective black-box optimization and build InferOpt, a reusable search framework that requires only variable bounds, a deterministic resource cost, and an evaluation hook. InferOpt searches on a frozen sampled proxy set, rejects over-budget candidates before any model call, tightens the budget adaptively, and re-validates Pareto representatives on full-scale dataset. One pipeline covers a 28-dimensional continuous KV space (Qwen2.5-7B) and a 26-dimensional discrete MoE space (DeepSeek-V2-Lite). On KV, post-prefill pruning cuts the 16K cache by 64.4% and TPOT by 22.9--48.5%, and the searched layer-wise budget by InferOpt beats a matched uniform budget by 7.3% and 14.0% of the Full KV reference points. On MoE, a searched top-k schedule by InferOpt removes 43.0% of routed token--expert pairs while staying within 0.59 points of the default, closer than the matched-budget baselines. Against Random Search, NSGA-II, and MOTPE, InferOpt leads on both spaces, taking the best proxy hypervolume and the lowest retention on KV and staying closest to the uncompressed reference at the lowest experts on MoE. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.04473 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-06T05:24:05.478Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).