Metraipolis › archive › dc_8ecc9727cceb

FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution — arXiv:2608.16157

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.16157
License (as archived): CC0-1.0
Archived: 2026-10-18T19:02:07.012Z
SHA-256 of the archived text: db4f1762f65a42db8c03a85f6e3b49b164e87b4138f30c9108a9f4161ada2339
Stable URL of this page: https://metraipolis.com/documento/dc_8ecc9727cceb

How to cite

FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution — arXiv:2608.16157 — arXiv (en), https://arxiv.org/abs/2608.16157 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8ecc9727cceb (Metraipolis archive dc_8ecc9727cceb, sha256 db4f1762f65a42db8c03a85f6e3b49b164e87b4138f30c9108a9f4161ada2339)

BibTeX

@misc{dc8ecc9727cceb,
  title = {FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution — arXiv:2608.16157},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.16157},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8ecc9727cceb; sha256 db4f1762f65a42db8c03a85f6e3b49b164e87b4138f30c9108a9f4161ada2339},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.16157. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Frontier open-weight models are increasingly available, but serving them still largely assumes datacenter infrastructure. We present FreeToken, an edge-native MoE serving system that treats a personal machine not as a small GPU, but as a unified, elastic inference platform. FreeToken co-designs the full serving stack, including model layout and loading, expert residency, CPU--GPU execution, agentic state reuse, and runtime memory management, around two realities of local AI: agent workloads continuously change their execution pattern, and edge hardware exposes heterogeneous resources whose balance differs from machine to machine. Rather than committing to a fixed offloading strategy, FreeToken continuously maps computation and model state onto the resources actually available. FreeToken supports more than 20 MoE models and real coding and tool-using agents across hardware ranging from an 8GB laptop GPU to a single workstation GPU. More importantly, it changes what these machines can practically serve, from a 35B model on a laptop to a 284B model on a gaming desktop and the 753B GLM-5.2 on a single workstation GPU. FreeToken turns open weights into deployable local software, making the machines users already own a practical platform for frontier-scale intelligence. We release the system at this http URL .

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-08-18 in cs.DC).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2608.16157 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T18:07:16.076Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).