Metraipolis › archive › dc_c25a5bb97190

A Study of LLMs' Preferences for Libraries and Programming Languages — arXiv:2503.17181

Source
arXiv (en)
Original
https://arxiv.org/abs/2503.17181
License (as archived)
CC0-1.0
Archived
2026-10-25T21:27:27.514Z
SHA-256 of the archived text
a7d20cc6d298f74a12ee452ae495c9f5fde5886411ea0691a9920b7e78bae843
Stable URL of this page
https://metraipolis.com/documento/dc_c25a5bb97190

How to cite

A Study of LLMs' Preferences for Libraries and Programming Languages — arXiv:2503.17181 — arXiv (en), https://arxiv.org/abs/2503.17181 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c25a5bb97190 (Metraipolis archive dc_c25a5bb97190, sha256 a7d20cc6d298f74a12ee452ae495c9f5fde5886411ea0691a9920b7e78bae843)
BibTeX
@misc{dcc25a5bb97190,
  title = {A Study of LLMs' Preferences for Libraries and Programming Languages — arXiv:2503.17181},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2503.17181},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c25a5bb97190; sha256 a7d20cc6d298f74a12ee452ae495c9f5fde5886411ea0691a9920b7e78bae843},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2503.17181. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Despite the rapid progress of large language models (LLMs) in code generation, existing evaluations focus on functional correctness or syntactic validity, overlooking how LLMs make critical design choices such as which library or programming language to use. To fill this gap, we perform the first empirical study of LLMs' preferences for libraries and programming languages when generating code, covering eight diverse LLMs. We observe a strong tendency to overuse widely adopted libraries such as NumPy; in up to 45% of cases, this usage is not required and deviates from the ground-truth solutions. The LLMs we study also show a significant preference toward Python as their default language. For high-performance project initialisation tasks where Python is not the optimal language, it remains the dominant choice in 58% of cases, and Rust is not used once. These results highlight how LLMs prioritise familiarity and popularity over suitability and task-specific optimality; underscoring the need for targeted fine-tuning, data diversification, and evaluation benchmarks that explicitly measure language and library selection fidelity.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-07 in cs.CL).
Categorie arXiv: Software Engineering (cs.SE) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2503.17181 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-25T21:24:38.235Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-07?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).