Metraipolis › archive › dc_7d2070cb23e4
Studying quantization trade-offs for efficient inference deployment in machine translation (2026) · OpenAlex W7172326854
- Source
- OpenAlex
- Original
- https://doi.org/10.48550/arxiv.2607.29397
- License (as archived)
- CC-BY-4.0
- Archived
- 2026-10-08T03:10:52.535Z
- SHA-256 of the archived text
c9fe991a52b01883b515d422a45b501dd10a25446d3c0ac35856ae73811c04e5- Stable URL of this page
https://metraipolis.com/documento/dc_7d2070cb23e4
How to cite
Studying quantization trade-offs for efficient inference deployment in machine translation (2026) · OpenAlex W7172326854 — OpenAlex, https://doi.org/10.48550/arxiv.2607.29397 (CC-BY-4.0); archived copy: https://metraipolis.com/documento/dc_7d2070cb23e4 (Metraipolis archive dc_7d2070cb23e4, sha256 c9fe991a52b01883b515d422a45b501dd10a25446d3c0ac35856ae73811c04e5)
BibTeX
@misc{dc7d2070cb23e4,
title = {Studying quantization trade-offs for efficient inference deployment in machine translation (2026) · OpenAlex W7172326854},
howpublished = {OpenAlex},
url = {https://doi.org/10.48550/arxiv.2607.29397},
note = {License CC-BY-4.0; archived copy https://metraipolis.com/documento/dc_7d2070cb23e4; sha256 c9fe991a52b01883b515d422a45b501dd10a25446d3c0ac35856ae73811c04e5},
year = {2026}
}Archived text
Reproduced under CC-BY-4.0. Attribution: OpenAlex, https://doi.org/10.48550/arxiv.2607.29397. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Opera scientifica in accesso aperto: scheda dai metadati OpenAlex (preprint).
Titolo: Studying quantization trade-offs for efficient inference deployment in machine translation
Autori: Jim Zhao, Sohir Maskey, Koen Oostermeijer, Douglas Orr, Teryn Jones
Anno: 2026 · Sede: arXiv (Cornell University)
DOI: https://doi.org/10.48550/arxiv.2607.29397
Accesso aperto: green · testo: https://arxiv.org/abs/2607.29397 · licenza del testo: cc-by
Citazioni ricevute (OpenAlex): 0
Temi (OpenAlex): Advanced Neural Network Applications; Natural Language Processing Techniques; Topic Modeling
Abstract (testo degli autori, licenza cc-by):
Deploying large language models in realistic server environments poses challenges, as the system needs to provide high-quality responses with low latency. Quantization is a common approach to reduce the memory footprint and improve inference efficiency, yet its impact on latency and throughput is rarely evaluated under controlled, orchestration-level workloads. In this work we study the quantization trade-offs of EuroLLM \citep{martins2025eurollm} across three model sizes ranging from 1.7B to 22B for efficient deployment on a single A100 or H100 GPU. We demonstrate that combining a document-chunking strategy with W4A8 or W8A8 quantization improves the latency-throughput Pareto-curve under a wide range of workloads. Furthermore, since standard machine translation (MT) benchmarks rely on isolated sentences and fail to capture long-context dynamics, we introduce a document-level evaluation based on DocHPLT \cite{o2025dochplt} to assess how text chunking strategies affect translation quality under quantization. Our results indicate that standard segment-level evaluation can potentially underestimate the interaction between quantization and long-context document translation, for some quantization formats, translation direction and models. Overall, our experiments show that the trade-off between inference efficiency and translation quality depends not only on the quantization format, but also on the choice of text chunking strategy.
Fonte: https://openalex.org/W7172326854 — metadati OpenAlex (CC0 1.0), consultati il 2026-10-07 via api.openalex.org. Interrogazione: «efficient inference quantization».