Metraipolis › archive › dc_9afcf985b523

How Weight Encoding Affects Language Model Placement and Performance on the Apple Neural Engine — arXiv:2608.22110

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.22110
License (as archived)
CC0-1.0
Archived
2026-10-18T07:56:10.855Z
SHA-256 of the archived text
c2154bd0a2f03499f14178a6b57e08359be94f36bbb3f0408bf8ac61b3e34ef8
Stable URL of this page
https://metraipolis.com/documento/dc_9afcf985b523

How to cite

How Weight Encoding Affects Language Model Placement and Performance on the Apple Neural Engine — arXiv:2608.22110 — arXiv (en), https://arxiv.org/abs/2608.22110 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_9afcf985b523 (Metraipolis archive dc_9afcf985b523, sha256 c2154bd0a2f03499f14178a6b57e08359be94f36bbb3f0408bf8ac61b3e34ef8)
BibTeX
@misc{dc9afcf985b523,
  title = {How Weight Encoding Affects Language Model Placement and Performance on the Apple Neural Engine — arXiv:2608.22110},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.22110},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_9afcf985b523; sha256 c2154bd0a2f03499f14178a6b57e08359be94f36bbb3f0408bf8ac61b3e34ef8},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.22110. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Weight compression can alter accelerator placement as well as memory traffic, complicating the interpretation of inference speedups. We investigate this interaction on the Apple Neural Engine through the public Core ML deployment path. Five independently trained language-model checkpoints span two architectures and dense fp16, int8, and ternary weights encoded with two-bit lookup tables. We combine compiler device plans, synchronized memory-controller measurements, and compute-unit exclusion controls for a fixed single-token forward workload. On an M1, the smaller fp16 export executes on the CPU despite permitting ANE execution, whereas its compressed counterparts exhibit ANE activity. The int8 export reduces warm forward latency by a factor of 1.9. The larger fp16 export also uses the ANE, indicating that dense encoding alone does not determine placement. Separate M3 energy measurements support the same direction of change. These results establish encoding-dependent placement in the measured deployment stack and show that compression comparisons require joint measurement of backend selection, latency, and traffic.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-08-25 in cs.AR).
Categorie arXiv: Machine Learning (cs.LG) ; Hardware Architecture (cs.AR); Performance (cs.PF).
Pagina del lavoro: https://arxiv.org/abs/2608.22110 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T07:48:03.873Z dalla pagina https://arxiv.org/catchup/cs.AR/2026-08-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).