Metraipolis › archive › dc_eaad7eed82b3

Beyond Correctness: How Novice Software Engineering Students Exercise Oversight of Generative AI — arXiv:2610.32118

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.32118
License (as archived)
CC0-1.0
Archived
2026-10-26T20:08:26.087Z
SHA-256 of the archived text
0e936644f55000868db6465b6da21aa11217b246a4e1855ef0d5c9b0eefec0c9
Stable URL of this page
https://metraipolis.com/documento/dc_eaad7eed82b3

How to cite

Beyond Correctness: How Novice Software Engineering Students Exercise Oversight of Generative AI — arXiv:2610.32118 — arXiv (en), https://arxiv.org/abs/2610.32118 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_eaad7eed82b3 (Metraipolis archive dc_eaad7eed82b3, sha256 0e936644f55000868db6465b6da21aa11217b246a4e1855ef0d5c9b0eefec0c9)
BibTeX
@misc{dceaad7eed82b3,
  title = {Beyond Correctness: How Novice Software Engineering Students Exercise Oversight of Generative AI — arXiv:2610.32118},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.32118},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_eaad7eed82b3; sha256 0e936644f55000868db6465b6da21aa11217b246a4e1855ef0d5c9b0eefec0c9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.32118. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Generative AI (GenAI) is shifting software engineering students' work from producing artefacts towards evaluating and supervising AI-generated outputs. However, little is known about what characteristics students identify in outputs when reporting interventions, or how they exercise oversight during authentic software engineering work. We analysed 962 student-AI interaction turns documented by 145 first-year software engineering students through structured AI Journals, including 172 student-declared interventions. Using a multi-dimensional coding framework, we characterised the output characteristics identified in these reported interventions, students' reasons for intervening, the actions they took, and how those actions were enacted. Fitness problems were identified in 74.4% of interventions, compared with 16.9% involving correctness problems, indicating that oversight more often concerned whether plausible AI outputs were suitable for the engineering task than whether they contained explicit errors. Among classifiable interventions, students most frequently modified generated outputs (51.0%) or refined their prompts (28.2%), with both response types commonly associated with engineering fit. Within-student analysis further showed that outputs perceived as containing minor flaws were associated with lower evaluation confidence than outputs perceived as perfect. Taken together, these findings characterise novice AI oversight as a process that extends beyond error detection to encompass both judgement of fitness for purpose and active intervention in AI-assisted engineering work.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.HC, nuovi lavori del giorno).
Categorie arXiv: Software Engineering (cs.SE) ; Human-Computer Interaction (cs.HC).
Pagina del lavoro: https://arxiv.org/abs/2610.32118 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-26T19:48:48.363Z dalla pagina https://arxiv.org/list/cs.HC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).