Metraipolis › archive › dc_cfd0f4c12f4e
Build or Adopt? Custom-Built vs. General-Purpose LLM Agent Harnesses for Automotive GUI Testing — arXiv:2610.33636
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.33636
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-27T08:00:22.961Z
- SHA-256 of the archived text
fdad46f17a4d941c54b2f678466a5cd2375b21f43a96a0de4059056c24bca8c7- Stable URL of this page
https://metraipolis.com/documento/dc_cfd0f4c12f4e
How to cite
Build or Adopt? Custom-Built vs. General-Purpose LLM Agent Harnesses for Automotive GUI Testing — arXiv:2610.33636 — arXiv (en), https://arxiv.org/abs/2610.33636 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cfd0f4c12f4e (Metraipolis archive dc_cfd0f4c12f4e, sha256 fdad46f17a4d941c54b2f678466a5cd2375b21f43a96a0de4059056c24bca8c7)
BibTeX
@misc{dccfd0f4c12f4e,
title = {Build or Adopt? Custom-Built vs. General-Purpose LLM Agent Harnesses for Automotive GUI Testing — arXiv:2610.33636},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.33636},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cfd0f4c12f4e; sha256 fdad46f17a4d941c54b2f678466a5cd2375b21f43a96a0de4059056c24bca8c7},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.33636. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
As general-purpose LLM agent harnesses mature, organizations must decide whether to continue developing in-house systems or adopt existing alternatives. We study this decision at Volvo Group for GUI testing of Android-based in-vehicle infotainment (IVI) systems. We compare a production, custom-built harness with two general-purpose harnesses, Claude Code and OpenCode, keeping the tools, accessed through the Model Context Protocol (MCP), and the models the same. We evaluate the harnesses with up to four models on Volvo's IVI test cases and on open-ended tasks in public apps (Google Maps, this http URL ), and interview ten practitioners: developers, the product owner, and testers. On Volvo's stepwise IVI tests, general-purpose harnesses match the custom-built harness in success but vary more across runs. Their inference cost per task is 2.6-10.7 times higher, yet the engineering effort they save outweighs this: for a nightly IVI regression suite, total monthly cost, including engineering effort and inference, is 3-66% lower. On open-ended public-app tasks, they succeed more often, mainly because they recover better from incorrect actions. Interviews suggest that external advances can turn an in-house harness into technical debt even without deterioration. Adopting a general-purpose harness shifts maintenance from the agent loop to tools and instructions, but gives teams less control over what the model receives. Regardless of the harness, testers want to inspect the agent's actions and ensure that it follows the prescribed test scenario. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Software Engineering (cs.SE) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.33636 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-27T07:55:30.839Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).