Metraipolis › archive › dc_89dbe4c3004e
Reproducible Evaluation of MoE Expert Caching: Replay Semantics, Workload Contamination, and Operating Regimes — arXiv:2608.07911
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.07911
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T08:53:58.744Z
- SHA-256 of the archived text
99542d9b58bf2513b86670c787737de284d32ea6ad3427e123f7aaa815b7ac14- Stable URL of this page
https://metraipolis.com/documento/dc_89dbe4c3004e
How to cite
Reproducible Evaluation of MoE Expert Caching: Replay Semantics, Workload Contamination, and Operating Regimes — arXiv:2608.07911 — arXiv (en), https://arxiv.org/abs/2608.07911 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_89dbe4c3004e (Metraipolis archive dc_89dbe4c3004e, sha256 99542d9b58bf2513b86670c787737de284d32ea6ad3427e123f7aaa815b7ac14)
BibTeX
@misc{dc89dbe4c3004e,
title = {Reproducible Evaluation of MoE Expert Caching: Replay Semantics, Workload Contamination, and Operating Regimes — arXiv:2608.07911},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.07911},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_89dbe4c3004e; sha256 99542d9b58bf2513b86670c787737de284d32ea6ad3427e123f7aaa815b7ac14},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.07911. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Mixture-of-Experts (MoE) models have outgrown accelerator memory, and offloading expert weights to host memory is now standard. This makes expert cache management an attractive lever: a policy that raised the hit rate would cut expert traffic per token. Evaluating that is a measurement problem, and we find the measurement fragile. With a trace-driven, event-atomic simulator over three MoE models (40, 64, 128 experts), we isolate three evaluation axes that change conclusions, not just numbers. Replay semantics: under a fused-event traffic contract, an inconsistent per-access replay inflates recency-based policies by 27-29% while leaving frequency-based and static ones within 4%, inverting the policy ranking. Workload contamination: probe sets using one instruction template per category produce verbatim-identical generation prefixes; a matched-pair rendering intervention moves the measured early-window effect by 19.4-31.9 points and reverses which workloads look most cache-friendly. Operating regimes: normalized miss fractions do not transfer across models, so the per-step expert union relative to per-layer capacity must be reported -- yet permuting only the temporal order of an identical event stream moves the offline-optimal gap from 44.9% to 30.8%, so it is not sufficient. Corrected, a stable gap to the offline optimum remains (44.2-45.9% over 13 frozen workload compositions). A forced-admission oracle attributes 84.3-96.6% of it to knowing which resident expert is used furthest in the future. A causal next-use predictor, used as an eviction rule, recovers -11.4% of the gap; it picks an optimal victim 3.4% of the time, against 2.4% for a random resident block and 20.6-22.1% for LRU and LFRU. Our position is narrow: in our evaluated settings a large offline-optimal gap substantially overstates the gains recovered by representative lightweight causal mechanisms. Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-08-11 in cs.PF). Categorie arXiv: Machine Learning (cs.LG) ; Performance (cs.PF). Pagina del lavoro: https://arxiv.org/abs/2608.07911 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T08:51:22.488Z dalla pagina https://arxiv.org/catchup/cs.PF/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).