Metraipolis › archive › dc_c03f9bbd08ca
Clustering-Based Balanced Sampling and Allocation with Data Parallelism for High-Performance Fine-Tuning — arXiv:2609.12584
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.12584
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T03:03:30.813Z
- SHA-256 of the archived text
8cecbde4669ccbcb52460b638ac685985f15952e63d132dd253702700b24d6ae- Stable URL of this page
https://metraipolis.com/documento/dc_c03f9bbd08ca
How to cite
Clustering-Based Balanced Sampling and Allocation with Data Parallelism for High-Performance Fine-Tuning — arXiv:2609.12584 — arXiv (en), https://arxiv.org/abs/2609.12584 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c03f9bbd08ca (Metraipolis archive dc_c03f9bbd08ca, sha256 8cecbde4669ccbcb52460b638ac685985f15952e63d132dd253702700b24d6ae)
BibTeX
@misc{dcc03f9bbd08ca,
title = {Clustering-Based Balanced Sampling and Allocation with Data Parallelism for High-Performance Fine-Tuning — arXiv:2609.12584},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.12584},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c03f9bbd08ca; sha256 8cecbde4669ccbcb52460b638ac685985f15952e63d132dd253702700b24d6ae},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.12584. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Instruction-tuning datasets for large language models (LLMs) are often large, redundant, and imbalanced, limiting efficient adaptation. Naive large-batch fine-tuning repeatedly includes overrepresented sample groups while weakly covering underrepresented but informative ones, especially under data parallelism (DP) across multiple GPUs. We propose CluSTER, a Cluster-aware balanced Sampling framework for Training Efficient data Reduction in DP instruction tuning. CluSTER curates a representative reduced dataset through gradient-space clustering and DP-aware balanced allocation, ensuring dual-level coverage across clusters and workers, while preserving the original data distribution by weighted update. As a result, CluSTER reduces redundant computation and improves training stability without compromising model quality. Across multiple instruction-tuning datasets, CluSTER reduces training time by up to 69.6% with almost no accuracy loss compared to prior sampling and data reduction methods. Code is available at this https URL . Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-14 in cs.AI). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.12584 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T02:51:27.927Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).