Metraipolis › archive › dc_2dfa2c82e90b

Beyond the Traceback: Using LLMs for Adaptive Explanations of Programming Errors — arXiv:2608.20896

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.20896
License (as archived)
CC0-1.0
Archived
2026-10-18T10:27:34.918Z
SHA-256 of the archived text
076fba53ee4859a83eb0b6fe5dc64138be9b9db94727439b79eadb4926c5c469
Stable URL of this page
https://metraipolis.com/documento/dc_2dfa2c82e90b

How to cite

Beyond the Traceback: Using LLMs for Adaptive Explanations of Programming Errors — arXiv:2608.20896 — arXiv (en), https://arxiv.org/abs/2608.20896 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2dfa2c82e90b (Metraipolis archive dc_2dfa2c82e90b, sha256 076fba53ee4859a83eb0b6fe5dc64138be9b9db94727439b79eadb4926c5c469)
BibTeX
@misc{dc2dfa2c82e90b,
  title = {Beyond the Traceback: Using LLMs for Adaptive Explanations of Programming Errors — arXiv:2608.20896},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.20896},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2dfa2c82e90b; sha256 076fba53ee4859a83eb0b6fe5dc64138be9b9db94727439b79eadb4926c5c469},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.20896. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Programming error messages are critical for software development, yet they remain difficult for novice programmers to interpret. While Large Language Models (LLMs) can rewrite these errors into clearer explanations, it remains unclear whether increased readability improves objective debugging performance or how explanation styles should align with programmer skill. We present a multi-stage crowdsourced study N=103 evaluating skill-targeted, LLM-generated Python error messages. Using a custom proficiency assessment, we categorized participants by skill level and tested standard interpreter messages against two LLM-generated styles: pragmatic (action-oriented) and contingent (scaffolded explanations). We measured both objective debugging metrics (fix rate, attempts, time-to-fix) and subjective perceptions (readability, cognitive load, tone). Our results show that while LLM-rewritten messages significantly improved subjective evaluations, with pragmatic messages rated as clearer and less cognitively demanding, these perceived gains did not translate into statistically significant improvements in objective debugging performance. This highlights a critical human-AI complementarity gap: explanations that feel better to users do not necessarily make them more effective debuggers. We discuss design implications for adaptive AI feedback systems, arguing that future tools should pivot from static skill-targeted rewriting toward dynamic adjustments based on a user's real-time repair trajectory.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-24 in cs.HC).
Categorie arXiv: Software Engineering (cs.SE) ; Human-Computer Interaction (cs.HC).
Pagina del lavoro: https://arxiv.org/abs/2608.20896 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T10:27:20.518Z dalla pagina https://arxiv.org/catchup/cs.HC/2026-08-24?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).