Metraipolis › archive › dc_5bae47cd6e50
The Vocabulary of Flaky Tests in Swift — arXiv:2609.25516
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.25516
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T15:16:05.286Z
- SHA-256 of the archived text
e90dcdef47f9cdfed0403976a5e586339569efa61edc61e970d69711cccb25c6- Stable URL of this page
https://metraipolis.com/documento/dc_5bae47cd6e50
How to cite
The Vocabulary of Flaky Tests in Swift — arXiv:2609.25516 — arXiv (en), https://arxiv.org/abs/2609.25516 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5bae47cd6e50 (Metraipolis archive dc_5bae47cd6e50, sha256 e90dcdef47f9cdfed0403976a5e586339569efa61edc61e970d69711cccb25c6)
BibTeX
@misc{dc5bae47cd6e50,
title = {The Vocabulary of Flaky Tests in Swift — arXiv:2609.25516},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.25516},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5bae47cd6e50; sha256 e90dcdef47f9cdfed0403976a5e586339569efa61edc61e970d69711cccb25c6},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.25516. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Flaky tests produce non-deterministic outcomes without code change, eroding CI confidence and delaying deliveries. While vocabulary-based machine learning prediction has proven effective for Java and JavaScript, no study has evaluated it for Swift, a language whose testing style is dominated by UI and asynchronous code. We collect 91 flaky and 22,349 stable tests from 15 open-source Swift projects via re-execution and commit-history mining, then train five classifiers (Random Forest, Decision Tree, Naive Bayes, SVM, KNN) on TF-IDF unigram+bigram features under stratified 5-fold cross-validation. Random Forest achieves the best performance (Precision = 0.92, F1 = 0.86, AUC = 0.95) and substantially outperforms trivial baselines, among them a vocabulary-threshold rule applied to the most informative tokens, confirming a genuine discriminative signal (MCC = 0.75 vs. 0.08 for the best baseline). Information-gain analysis reveals two complementary signal types. Flakiness markers appear predominantly in unstable tests and comprise concurrency primitives (async, await), expectation-based synchronisation (expectation, fulfill), error propagation (throws), and explicit timing dependence (timeout, wait, now). Stability markers, chiefly the assertion vocabulary of plainly synchronous tests (xctassertequal), count as evidence against flakiness. Error analysis shows that the model fails when flakiness is hidden in shared infrastructure outside the test body or when async constructs are used in a deterministic context, exposing the intrinsic limit of lexical prediction. These results extend vocabulary-based flakiness detection to the Swift ecosystem and characterise both its effectiveness and its boundaries. Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-23 in cs.SE). Categorie arXiv: Software Engineering (cs.SE). Pagina del lavoro: https://arxiv.org/abs/2609.25516 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T14:42:48.163Z dalla pagina https://arxiv.org/catchup/cs.SE/2026-09-23?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).