Metraipolis › archive › dc_e7364a3a210b
FactorSpec: Serving Exact Relational Constraints as Speculative Drafts — arXiv:2611.02297
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.02297
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T10:28:09.191Z
- SHA-256 of the archived text
e9aa5f68ed7c787bb622d590b5c9414744d41a0d9d798df6e1e0c58fe0a33b3c- Stable URL of this page
https://metraipolis.com/documento/dc_e7364a3a210b
How to cite
FactorSpec: Serving Exact Relational Constraints as Speculative Drafts — arXiv:2611.02297 — arXiv (en), https://arxiv.org/abs/2611.02297 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e7364a3a210b (Metraipolis archive dc_e7364a3a210b, sha256 e9aa5f68ed7c787bb622d590b5c9414744d41a0d9d798df6e1e0c58fe0a33b3c)
BibTeX
@misc{dce7364a3a210b,
title = {FactorSpec: Serving Exact Relational Constraints as Speculative Drafts — arXiv:2611.02297},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.02297},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e7364a3a210b; sha256 e9aa5f68ed7c787bb622d590b5c9414744d41a0d9d798df6e1e0c58fe0a33b3c},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02297. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Speculative decoding speeds up LLM serving by drafting tokens that the target model verifies in one pass, and its gain grows with the drafts the target accepts. Structured outputs should be an ideal case, because much of a record follows from relations among its fields: an itinerary's days follow from its cities' stays and flights. Serving engines, however, draft such text by proposing it from the prompt or a learned head, and their grammar masks enforce only syntax, so the model must satisfy the relations, which it violates: on NATURAL PLAN's trip problems, Qwen3-8B writes a valid itinerary for 10%, and retries that name the violated relation raise this to 14%. We present FactorSpec, which serves exact relational constraints as speculative drafts. FactorSpec compiles a JSON Schema and its relations into a factor graph with an exact feasibility check, drafts every token the constraint determines as certain, verifies each request at a width chosen for it, and computes the masks in worker processes outside the scheduler while the GPU runs. In SGLang, its drafts yield up to 10.5 tokens per verify step, up to 1.9x those of the fastest n-gram drafter, and it decodes up to 1.43x faster than speculators tuned per load, at a grammar mask's cost; it also leads in vLLM. Because every record is also valid, it serves up to 13.9x the valid records per GPU-second of syntax-only baselines. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2611.02297 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T10:19:50.166Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).