Metraipolis › archive › dc_dcc7dd941168
ACSeL: Adaptive Configuration Search for Hardware-Constrained LLM Inference — arXiv:2611.07570
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.07570
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-06T14:50:35.534Z
- SHA-256 of the archived text
23690cf50dcc12276614baad829fa583fbebd8f13166b27885ac5a513a453017- Stable URL of this page
https://metraipolis.com/documento/dc_dcc7dd941168
How to cite
ACSeL: Adaptive Configuration Search for Hardware-Constrained LLM Inference — arXiv:2611.07570 — arXiv (en), https://arxiv.org/abs/2611.07570 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dcc7dd941168 (Metraipolis archive dc_dcc7dd941168, sha256 23690cf50dcc12276614baad829fa583fbebd8f13166b27885ac5a513a453017)
BibTeX
@misc{dcdcc7dd941168,
title = {ACSeL: Adaptive Configuration Search for Hardware-Constrained LLM Inference — arXiv:2611.07570},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.07570},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dcc7dd941168; sha256 23690cf50dcc12276614baad829fa583fbebd8f13166b27885ac5a513a453017},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.07570. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Serving large language models (LLMs) under tight memory and latency service-level objectives (SLOs), including on edge devices, requires jointly selecting the model size, batch size, quantization algorithm, and mixed-precision quantization recipes. Exhaustive search over this combinatorially large space is infeasible. Unlike data-center serving, which often prioritizes throughput or generation speed, resource-constrained deployments typically set memory and latency budgets as constraints and then seek the configuration with the highest generation quality among feasible ones. To this end, we propose ACSeL, a framework that identifies configurations predicted to satisfy deployment constraints while achieving high predicted generation quality. To enable efficient quality estimation, we use Relative Model Quality Degradation (RMQD), a proxy based on the divergence between the candidate and reference models' next-token distributions over a small calibration corpus. Across the evaluated model sizes and precisions, RMQD is inexpensive to evaluate and strongly correlated with downstream benchmark performance. ACSeL combines (i) a lightweight quality surrogate that predicts RMQD directly from a configuration; (ii) Few-shot Median Calibration (FMC), which calibrates operator-profile-based latency estimates using a small number of end-to-end profiles; and (iii) a structured search algorithm BandPareto for exploring layerwise mixed-precision configurations. Experiments on an NVIDIA RTX A6000 workstation and an NVIDIA Jetson Thor edge platform show that FMC substantially improves deployment feasibility, while BandPareto + FMC achieves the highest feasibility and the lowest RMQD gaps to the best measured configurations in the evaluation set. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2611.07570 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-06T14:21:42.628Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).