Metraipolis › archive › dc_e5edccacd8f6
GGUF-Metadata Prediction of Single-Sequence llama.cpp Throughput Across Three Systems — arXiv:2609.14864
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.14864
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T22:47:40.864Z
- SHA-256 of the archived text
21f0af905602447db52222bc4cffd5aa31376a8b40d1ea34bd340f65e5a77b12- Stable URL of this page
https://metraipolis.com/documento/dc_e5edccacd8f6
How to cite
GGUF-Metadata Prediction of Single-Sequence llama.cpp Throughput Across Three Systems — arXiv:2609.14864 — arXiv (en), https://arxiv.org/abs/2609.14864 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e5edccacd8f6 (Metraipolis archive dc_e5edccacd8f6, sha256 21f0af905602447db52222bc4cffd5aa31376a8b40d1ea34bd340f65e5a77b12)
BibTeX
@misc{dce5edccacd8f6,
title = {GGUF-Metadata Prediction of Single-Sequence llama.cpp Throughput Across Three Systems — arXiv:2609.14864},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.14864},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e5edccacd8f6; sha256 21f0af905602447db52222bc4cffd5aa31376a8b40d1ea34bd340f65e5a77b12},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.14864. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
We predict single-sequence model throughput from GGUF metadata using roofline-shaped predictors with quantization-specific scale factors fitted on reference models. The scored cohort comprises 318 phase-depth measurements from 53 host-file configurations on two Apple M4 Max systems and an NVIDIA RTX 5080. On host-specific held-out sets of four, five, and two configurations, an active-parameter decode model obtains 13.1%, 14.4%, and 36.1% mean absolute percentage error (MAPE), versus 49.4%, 55.3%, and 51.9% when charging total parameters. Leave-one-host-out coefficients fitted on the other two systems yield 11.6%, 16.8%, and 36.0% test MAPE. A low-bit model ladder changes ordering across runtime stacks. The P2 prefill baseline gives 18.7%, 22.2%, and 108.2% test MAPE. GGUF structure helps on all three systems, but fitted efficiencies are not universal. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-15 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG); Performance (cs.PF). Pagina del lavoro: https://arxiv.org/abs/2609.14864 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T22:29:03.233Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-15?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).