Metraipolis › archive › dc_7d255db99e08

Metamorphic Testing for Evaluating Robustness of Sentiment Analysis Models — arXiv:2611.02248

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.02248
License (as archived): CC0-1.0
Archived: 2026-11-03T10:57:47.220Z
SHA-256 of the archived text: a133a7e2c8acfa620eddf277871c850abf8fcc7da848bfbd93676f490a417e63
Stable URL of this page: https://metraipolis.com/documento/dc_7d255db99e08

How to cite

Metamorphic Testing for Evaluating Robustness of Sentiment Analysis Models — arXiv:2611.02248 — arXiv (en), https://arxiv.org/abs/2611.02248 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7d255db99e08 (Metraipolis archive dc_7d255db99e08, sha256 a133a7e2c8acfa620eddf277871c850abf8fcc7da848bfbd93676f490a417e63)

BibTeX

@misc{dc7d255db99e08,
  title = {Metamorphic Testing for Evaluating Robustness of Sentiment Analysis Models — arXiv:2611.02248},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.02248},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7d255db99e08; sha256 a133a7e2c8acfa620eddf277871c850abf8fcc7da848bfbd93676f490a417e63},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02248. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Machine learning systems are challenging to test using traditional software testing methods because their behavior is learned from data rather than explicitly programmed. This challenge is particularly significant in natural language processing (NLP), where small input variations can lead to unexpected model behavior. Conventional evaluation metrics, such as accuracy and F1-score, often fail to capture such inconsistencies. Metamorphic testing addresses this limitation by evaluating expected relationships between original and transformed inputs. With the increasing prevalence of generative artificial intelligence (GenAI) and language-based AI systems, the need for systematic robustness testing has become even more pronounced. Motivated by this growing need, this paper presents a lightweight, batch-based metamorphic testing approach for evaluating the robustness of such systems. In particular, we consider sentiment analysis as a representative NLP task and evaluate a pretrained classifier under four transformations: synonym replacement, typo injection, word-order variation, and negation. Using consistency score, flip accuracy, and confidence change, we identify strengths in handling semantic variations and vulnerabilities to textual noise and logical transformations. These findings highlight the value of metamorphic testing in identifying behavioral inconsistencies beyond conventional performance metrics and demonstrate its potential as a systematic framework for evaluating the robustness of language-based AI systems and guiding performance improvements before deployment.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2611.02248 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:49:38.633Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).