Metraipolis › archive › dc_a5388fd75bb3
BnBERT-iPET: Sparse Few-Shot Language Modeling for Bengali via Lottery Ticket Pruning — arXiv:2608.05104
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.05104
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-31T11:35:33.546Z
- SHA-256 of the archived text
37373e8cd0637015e663c01cde14f9f593a302209d0fdc5483d7dbbdca2a97b3- Stable URL of this page
https://metraipolis.com/documento/dc_a5388fd75bb3
How to cite
BnBERT-iPET: Sparse Few-Shot Language Modeling for Bengali via Lottery Ticket Pruning — arXiv:2608.05104 — arXiv (en), https://arxiv.org/abs/2608.05104 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a5388fd75bb3 (Metraipolis archive dc_a5388fd75bb3, sha256 37373e8cd0637015e663c01cde14f9f593a302209d0fdc5483d7dbbdca2a97b3)
BibTeX
@misc{dca5388fd75bb3,
title = {BnBERT-iPET: Sparse Few-Shot Language Modeling for Bengali via Lottery Ticket Pruning — arXiv:2608.05104},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.05104},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a5388fd75bb3; sha256 37373e8cd0637015e663c01cde14f9f593a302209d0fdc5483d7dbbdca2a97b3},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.05104. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Deep neural networks have shown impressive success in NLP tasks owing to their complex structure and huge number of edges. Achieving state-of-the-art performance in natural language processing with a large pre-trained model such as BERT is expensive and time-consuming, carries a large carbon footprint, and is difficult to realize on machines with minimal computational capability. This creates a barrier to training complex models for resource-constrained languages such as Bengali. However, in a complex neural model, not all edges are equally impactful, and the contributions of some of them can be neglected. Pruning promises to reduce the memory footprint of regular networks, shorten the training time of ever-growing networks, and increase inference efficiency without sacrificing comparable performance. In this work, we introduce BnBERT-iPET, a sparse few-shot language modeling approach for Bengali, and experimentally show that a lightweight few-shot-learned language model retaining only 10% of the edges of an initial model such as BERT can perform neck and neck with much larger models on challenging tasks for a resource-constrained language such as Bengali. By learning from few shots through iterative pattern exploiting training and achieving 90% sparsity with the Lottery Ticket Hypothesis pruning technique, our pruned BnBERT-iPET model proves to be a tough competitor to state-of-the-art language models such as Bangla Electra, Indic-BERT, and XLM-RoBERTa on downstream tasks over standard benchmark datasets of the Bengali language. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-06 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2608.05104 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-31T11:24:48.810Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-06?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).