Metraipolis › archive › dc_eef744460df6

Interface Quilt: Modifying Applications with Computer Use Agents and Generative UI — arXiv:2611.01891

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.01891
License (as archived)
CC0-1.0
Archived
2026-11-03T11:18:46.476Z
SHA-256 of the archived text
142dd67a060b9520644f6021ee320345554bcef2966032a599750db80d592a34
Stable URL of this page
https://metraipolis.com/documento/dc_eef744460df6

How to cite

Interface Quilt: Modifying Applications with Computer Use Agents and Generative UI — arXiv:2611.01891 — arXiv (en), https://arxiv.org/abs/2611.01891 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_eef744460df6 (Metraipolis archive dc_eef744460df6, sha256 142dd67a060b9520644f6021ee320345554bcef2966032a599750db80d592a34)
BibTeX
@misc{dceef744460df6,
  title = {Interface Quilt: Modifying Applications with Computer Use Agents and Generative UI — arXiv:2611.01891},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.01891},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_eef744460df6; sha256 142dd67a060b9520644f6021ee320345554bcef2966032a599750db80d592a34},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01891. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Modifying an existing application is difficult when it does not provide programmatic access for that change. Recent advances in computer-use agents (CUAs) enable richer understanding and operation of existing interfaces, while generative UI (GenUI) makes it practical to generate new ones. We examine how these advances expand software modification through a design space of what can be changed and a framework for how modifications can be implemented. We then introduce Interface Quilt, a system that combines CUAs and GenUI to generate interface patches, which are new interfaces displayed over an existing application to modify its presentation, interaction, and capabilities. Through demonstrations across a variety of applications, we show that interface patches support a range of modifications without source code, APIs, or accessibility data.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.HC, nuovi lavori del giorno).
Categorie arXiv: Human-Computer Interaction (cs.HC).
Pagina del lavoro: https://arxiv.org/abs/2611.01891 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T11:02:22.166Z dalla pagina https://arxiv.org/list/cs.HC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).