Metraipolis › archive › dc_4e6cf3ab9f58

Domain-specific Pretraining Profile and Transformer Performance: Evidence from Modeling Digital Pragmatics in Arabic-English Code-switching — arXiv:2609.14571

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.14571
License (as archived)
CC0-1.0
Archived
2026-10-15T23:15:20.126Z
SHA-256 of the archived text
e4de8817d4e22738bec749c0360fe698e14460aa6d2e2b7617468039b9a2f627
Stable URL of this page
https://metraipolis.com/documento/dc_4e6cf3ab9f58

How to cite

Domain-specific Pretraining Profile and Transformer Performance: Evidence from Modeling Digital Pragmatics in Arabic-English Code-switching — arXiv:2609.14571 — arXiv (en), https://arxiv.org/abs/2609.14571 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4e6cf3ab9f58 (Metraipolis archive dc_4e6cf3ab9f58, sha256 e4de8817d4e22738bec749c0360fe698e14460aa6d2e2b7617468039b9a2f627)
BibTeX
@misc{dc4e6cf3ab9f58,
  title = {Domain-specific Pretraining Profile and Transformer Performance: Evidence from Modeling Digital Pragmatics in Arabic-English Code-switching — arXiv:2609.14571},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.14571},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4e6cf3ab9f58; sha256 e4de8817d4e22738bec749c0360fe698e14460aa6d2e2b7617468039b9a2f627},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.14571. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

This study highlights the role of domain-specific pretraining profile (DSPP) in Transformer performance for modeling digital pragmatics in Arabic-English code-switched discourse. It evaluates MARBERT and XLM-R(oBERTa), with BERT serving as a general-purpose baseline. The models were evaluated on their ability to classify context-sensitive pragmatic functions in code-switched social-media discourse. 11695 unique X posts were collected via Python and utilized for the study. The study employs a quantitative and qualitative NLP approach, following a supervised pipeline. Findings unveil that MARBERT consistently surpasses XLM-R with validation Macro F1 increasing from 0.39 to 0.84 and validation loss decreasing from 0.55 to 0.19. On an independent test set, it achieved 0.96 accuracy, 0.83 macro precision, 0.87 macro recall, and 0.85 Macro F1, while XLM-R achieved 0.92 test accuracy but a substantially lower Macro F1 of 0.52. This was also supported by class-level performance where MARBERT outperforms XLM-R considerably with F1 improvements ranging from +0.33 to +0.60, demonstrating a clear advantage in modeling Arabic digital pragmatics. The study concludes that Transformer performance depends more on DSPP than multilingual coverage alone, as the latter does not guarantee optimal performance on a highly specialized pragmatic classification task.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-15 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.14571 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T22:51:09.896Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-15?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).