Metraipolis › archive › dc_da74887badfb
Vosti: Specifying, Implementing, and Verifying Deterministic LLM Inference — arXiv:2609.38981
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.38981
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T13:14:49.645Z
- SHA-256 of the archived text
34da34c02b7143e486dab8e960e0850b95177b737a2974a66d04cf5bbb1c03a4- Stable URL of this page
https://metraipolis.com/documento/dc_da74887badfb
How to cite
Vosti: Specifying, Implementing, and Verifying Deterministic LLM Inference — arXiv:2609.38981 — arXiv (en), https://arxiv.org/abs/2609.38981 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_da74887badfb (Metraipolis archive dc_da74887badfb, sha256 34da34c02b7143e486dab8e960e0850b95177b737a2974a66d04cf5bbb1c03a4)
BibTeX
@misc{dcda74887badfb,
title = {Vosti: Specifying, Implementing, and Verifying Deterministic LLM Inference — arXiv:2609.38981},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.38981},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_da74887badfb; sha256 34da34c02b7143e486dab8e960e0850b95177b737a2974a66d04cf5bbb1c03a4},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.38981. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
LLM inference systems may vary batch composition, prompt chunking, prefill/decode execution, and KV-cache reuse, eviction, or recomputation. These optimizations should not affect system outputs. Production systems, including vLLM's batch-invariant mode and SGLang's deterministic mode, target this goal but lack a formal system-level specification. We formalize deterministic LLM inference: under a fixed model and deployment configuration, requests with the same prompt and initial sampler state produce bitwise-identical logits at corresponding output positions across executions. Our tests find that these production modes produce different logits under some execution variations. To address this limitation, we present Vosti, an inference engine designed and verified against this specification. Vosti chooses kernels independently of runtime engine state and ties cached KV values to their logical token prefixes. Its proof decomposes at the engine/GPU kernel boundary: a Verus inductive proof establishes that scheduling and the paged, prefix-sharing KV-cache preserve output logits, while a Triton analyzer proves bitwise-equal selected kernel outputs across batches, query lengths, and paged KV-cache layouts. Vosti produces bitwise-identical logits across every tested execution variation and achieves performance comparable to vLLM's batch-invariant mode on decode-heavy workloads, while providing a stronger, formally verified determinism guarantee. Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-10-01 in cs.DC). Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC) ; Programming Languages (cs.PL). Pagina del lavoro: https://arxiv.org/abs/2609.38981 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T12:46:39.712Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).