Metraipolis › archive › dc_385a04be4bc1

Reduced Matrix Multiplication: Input-Adaptive Matrix-Product Reduction for LLM Inference — arXiv:2608.13426

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.13426
License (as archived)
CC0-1.0
Archived
2026-10-19T04:18:30.910Z
SHA-256 of the archived text
da1bdfc35abb6a851e3962e0e180f454633245d9ea5b5f73da055965977c5354
Stable URL of this page
https://metraipolis.com/documento/dc_385a04be4bc1

How to cite

Reduced Matrix Multiplication: Input-Adaptive Matrix-Product Reduction for LLM Inference — arXiv:2608.13426 — arXiv (en), https://arxiv.org/abs/2608.13426 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_385a04be4bc1 (Metraipolis archive dc_385a04be4bc1, sha256 da1bdfc35abb6a851e3962e0e180f454633245d9ea5b5f73da055965977c5354)
BibTeX
@misc{dc385a04be4bc1,
  title = {Reduced Matrix Multiplication: Input-Adaptive Matrix-Product Reduction for LLM Inference — arXiv:2608.13426},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.13426},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_385a04be4bc1; sha256 da1bdfc35abb6a851e3962e0e180f454633245d9ea5b5f73da055965977c5354},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.13426. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Transformer-based language models achieve strong performance but incur substantial inference cost due to repeated high-dimensional matrix multiplications. We propose Reduced Matrix Multiplication (RMM), a training-free, input-adaptive inference method that reduces Transformer matrix products by selecting informative slices along their contraction dimensions, without modifying model weights. Under a simple retention-ratio control, RMM provides a smooth and predictable accuracy-efficiency trade-off. Across language models ranging from 1B to 70B parameters, we find that reduction tolerance depends on the model family, task, component, and retention ratio, although it often improves with model scale. Under moderate reduction, RMM remains robust across the evaluated discriminative, autoregressive generation, and long-context settings. We further show that the same principle extends to multimodal vision-language inference. Mechanistic ablations reveal a structural asymmetry within Transformers: attention-side computations are substantially more reducible than MLP components. Finally, wall-clock benchmarks with custom kernels on an NVIDIA A100 show that these computational savings can translate into practical runtime gains, especially at longer sequence lengths. Together, these results position RMM as a scalable direction for input-adaptive inference-time optimization.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-14 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.13426 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T04:13:06.766Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).