Metraipolis › archive › dc_d7aa61393e81

Revisiting the Performance of Generative Artificial Intelligence on Introductory Object-Oriented Programming Assessments: Insights from 2026 — arXiv:2608.16318

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.16318
License (as archived)
CC0-1.0
Archived
2026-10-18T18:07:09.206Z
SHA-256 of the archived text
ec286a0d4caf48ae999aaf5c30c1683644eda2c7c21103421ec78a7bac7112ad
Stable URL of this page
https://metraipolis.com/documento/dc_d7aa61393e81

How to cite

Revisiting the Performance of Generative Artificial Intelligence on Introductory Object-Oriented Programming Assessments: Insights from 2026 — arXiv:2608.16318 — arXiv (en), https://arxiv.org/abs/2608.16318 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d7aa61393e81 (Metraipolis archive dc_d7aa61393e81, sha256 ec286a0d4caf48ae999aaf5c30c1683644eda2c7c21103421ec78a7bac7112ad)
BibTeX
@misc{dcd7aa61393e81,
  title = {Revisiting the Performance of Generative Artificial Intelligence on Introductory Object-Oriented Programming Assessments: Insights from 2026 — arXiv:2608.16318},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.16318},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d7aa61393e81; sha256 ec286a0d4caf48ae999aaf5c30c1683644eda2c7c21103421ec78a7bac7112ad},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.16318. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Recent advances in Generative Artificial Intelligence (GenAI) have substantially improved the ability of large language models (LLMs) to generate and explain source code. However, their performance on authentic object-oriented programming (OOP) assessments remains insufficiently understood. This study evaluates five widely used GenAI systems, ChatGPT-5.2, DeepSeek-V3, Gemini 2.5 Flash, Claude Sonnet 4.5, and M365 Copilot, using programming tests and examination tasks from an introductory university OOP course. The generated solutions were assessed using the same grading criteria applied to students and compared with historical student results from the same course, as well as findings from the previous year. Common errors were also analyzed to identify recurring limitations across models. All evaluated GenAI systems achieved higher scores than the average student cohort and frequently obtained full marks on longer programming tasks. Nevertheless, they occasionally produced non-compiling code and continued to struggle with advanced OOP concepts, particularly interfaces, abstract classes, and certain inheritance-related tasks. Performance was also limited on graphics-related questions involving image interpretation. Compared with the previous year, the evaluated systems demonstrated noticeable improvements across most assessments while exhibiting several recurring error patterns. The findings provide an updated evaluation of the capabilities and limitations of contemporary GenAI systems on authentic introductory OOP assessments. They also offer evidence that can inform the design of programming assessments, the responsible integration of GenAI tools into software engineering education, and future studies evaluating the evolution of AI-assisted programming.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.AI).
Categorie arXiv: Software Engineering (cs.SE) ; Artificial Intelligence (cs.AI); Performance (cs.PF).
Pagina del lavoro: https://arxiv.org/abs/2608.16318 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T16:29:14.105Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).