Metraipolis › archive › dc_833d7899b299

Beyond Task Completion: Measuring Interaction Cost in Terminal User Interfaces — arXiv:2610.05047

Source
arXiv
Original
https://arxiv.org/abs/2610.05047
License (as archived)
CC0-1.0
Archived
2026-10-06T05:32:41.670Z
SHA-256 of the archived text
b6c639726cb0eb301e27cee6c5e8eeea6488bc108b75b68ad230dca6bc61be08
Stable URL of this page
https://metraipolis.com/documento/dc_833d7899b299

How to cite

Beyond Task Completion: Measuring Interaction Cost in Terminal User Interfaces — arXiv:2610.05047 — arXiv, https://arxiv.org/abs/2610.05047 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_833d7899b299 (Metraipolis archive dc_833d7899b299, sha256 b6c639726cb0eb301e27cee6c5e8eeea6488bc108b75b68ad230dca6bc61be08)
BibTeX
@misc{dc833d7899b299,
  title = {Beyond Task Completion: Measuring Interaction Cost in Terminal User Interfaces — arXiv:2610.05047},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.05047},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_833d7899b299; sha256 b6c639726cb0eb301e27cee6c5e8eeea6488bc108b75b68ad230dca6bc61be08},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.05047. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) are increasingly used through terminal user interfaces (TUIs), yet task completion alone does not capture how difficult an interface is to understand and operate. Existing human assessments and LLM-generated ratings or reports do not provide repeatable measurements of interaction effort grounded in verified task execution. We propose Agent-as-a-User, an evaluation paradigm that places an LLM agent in the user role. Agent-KLM separates agent-side interpretation and operation costs and relates them structurally to human interaction. TUINaut operationalizes the paradigm by recording interaction trajectories and verifying outcomes with actor-independent oracles. Using TUINaut-Bench, we study 84 tasks across 15 real-world TUIs with six human participants and five observation-evaluator configurations, and evaluate 45 TUIs generated by three leading stacks. Similar aggregate success rates mask differences in which tasks humans and agents complete, whereas interpretation and operation costs follow correlated task rankings, especially for operation. This pattern persists across configurations. Generated TUIs can implement correct functionality while still requiring usability improvements. These results position Agent-as-a-User as a repeatable, execution-grounded paradigm for measuring task-based usability.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.SE, nuovi lavori del giorno).
Categorie arXiv: Software Engineering (cs.SE).
Pagina del lavoro: https://arxiv.org/abs/2610.05047 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T05:27:20.399Z dalla pagina https://arxiv.org/list/cs.SE/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).