Metraipolis › archive › dc_d4c6d42ecc47

Hibana: Fast Agentic Inference on Unified-Memory Devices — arXiv:2611.10469

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.10469
License (as archived)
CC0-1.0
Archived
2026-11-10T18:42:31.536Z
SHA-256 of the archived text
ce0b77b324284aa8a269ce78591857b6f35acee6e813797a7d58fefc3aab867c
Stable URL of this page
https://metraipolis.com/documento/dc_d4c6d42ecc47

How to cite

Hibana: Fast Agentic Inference on Unified-Memory Devices — arXiv:2611.10469 — arXiv (en), https://arxiv.org/abs/2611.10469 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d4c6d42ecc47 (Metraipolis archive dc_d4c6d42ecc47, sha256 ce0b77b324284aa8a269ce78591857b6f35acee6e813797a7d58fefc3aab867c)
BibTeX
@misc{dcd4c6d42ecc47,
  title = {Hibana: Fast Agentic Inference on Unified-Memory Devices — arXiv:2611.10469},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.10469},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d4c6d42ecc47; sha256 ce0b77b324284aa8a269ce78591857b6f35acee6e813797a7d58fefc3aab867c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.10469. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Local AI promises capable personal assistants that run on users' own devices, keeping data and computation under their control. Open-weight models that fit on local devices can now handle practical tasks such as coding. Yet current inference engines fall short. Popular engines such as vLLM and SGLang target datacenter GPUs with dedicated host memory, while local engines underuse the memory hierarchy of unified-memory devices. We present Hibana, an inference engine for unified-memory devices that co-optimizes model representation, memory management, and decoding strategy. Model surgery, the process of editing a trained model's architecture to suit its hardware, changes how the model computes, shrinking its memory footprint and data movement. Unified-memory management preserves each agent's state across turns, so long-running agents can resume instantly instead of recomputing their state. Adaptive speculation generates several tokens per step. Hibana codesigns these features around the shared memory capacity and limited bandwidth of unified-memory systems-on-chip to support local multi-agent workloads. On a single DGX Spark serving Qwen and Nemotron models of up to 125B parameters, Hibana completes up to 3.4$\times$ more agentic tasks per hour and delivers up to 3.9$\times$ higher decoding throughput than other established frameworks such as vLLM, NVIDIA Flare, this http URL , and TensorFold. Code is available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.10469 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-10T18:37:37.590Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).