Metraipolis › archive › dc_e024bf2162ac

Evaluating Code Recommender Systems: A Review — arXiv:2609.30351

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.30351
License (as archived): CC0-1.0
Archived: 2026-10-08T02:45:09.300Z
SHA-256 of the archived text: 318a238397dfc90d7da0ddb303c88be2c2bec9fa08995145811fd0164c85eada
Stable URL of this page: https://metraipolis.com/documento/dc_e024bf2162ac

How to cite

Evaluating Code Recommender Systems: A Review — arXiv:2609.30351 — arXiv (en), https://arxiv.org/abs/2609.30351 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e024bf2162ac (Metraipolis archive dc_e024bf2162ac, sha256 318a238397dfc90d7da0ddb303c88be2c2bec9fa08995145811fd0164c85eada)

BibTeX

@misc{dce024bf2162ac,
  title = {Evaluating Code Recommender Systems: A Review — arXiv:2609.30351},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.30351},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e024bf2162ac; sha256 318a238397dfc90d7da0ddb303c88be2c2bec9fa08995145811fd0164c85eada},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.30351. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Context: Code recommender systems (CRSs) are specialized software systems operating on source artifacts to provide automatically generated recommendations to software developers in all phases of development. The goal is to improve software quality while enhancing developers' efficiency, effectiveness, and experience. Problem: Despite the growing importance, little is known about the state of evaluating these systems in a human-centric manner. Research Approach: We conducted a systematic literature review to identify and to synthesize primary studies evaluating CRSs (2017-2024). Ninety-two publications were included and subjected to a systematic content analysis. Results: Our study confirms that Offline Evaluations are the most common, system-centric evaluation type, whereas user-centric evaluations (Online Evaluations, User Studies) are rarely reported. Evaluations are concentrated on the Software Construction phase. Most studies explicitly report threats to validity, with External threats to study Materials being the most common. Conclusion: The state of evaluations on CRSs is narrowly focused on single or a few system qualities. There is an imbalance between a majority of system-centric evaluations and a minority of user-centric evaluations. Combined, multi-type evaluations are rarely reported.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-28 in cs.SE).
Categorie arXiv: Software Engineering (cs.SE).
Pagina del lavoro: https://arxiv.org/abs/2609.30351 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T02:09:54.632Z dalla pagina https://arxiv.org/catchup/cs.SE/2026-09-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).