Metraipolis › archive › dc_09e9b3ff2d1b

MemU-Bench: Benchmarking GUI Agents on Long-Horizon User Interactions — arXiv:2611.00191

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.00191
License (as archived)
CC0-1.0
Archived
2026-11-03T18:11:50.181Z
SHA-256 of the archived text
a036364eb6ab706396396cd142e00f6b4c9b9ab3acee52fd962a11af6cc17358
Stable URL of this page
https://metraipolis.com/documento/dc_09e9b3ff2d1b

How to cite

MemU-Bench: Benchmarking GUI Agents on Long-Horizon User Interactions — arXiv:2611.00191 — arXiv (en), https://arxiv.org/abs/2611.00191 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_09e9b3ff2d1b (Metraipolis archive dc_09e9b3ff2d1b, sha256 a036364eb6ab706396396cd142e00f6b4c9b9ab3acee52fd962a11af6cc17358)
BibTeX
@misc{dc09e9b3ff2d1b,
  title = {MemU-Bench: Benchmarking GUI Agents on Long-Horizon User Interactions — arXiv:2611.00191},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.00191},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_09e9b3ff2d1b; sha256 a036364eb6ab706396396cd142e00f6b4c9b9ab3acee52fd962a11af6cc17358},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.00191. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Existing Graphical User Interface (GUI) benchmarks predominantly rely on precise and stateless user instructions. In everyday usage, however, instructions are often \emph{vague and history-dependent}: users omit the exact details of their past operations because they no longer remember them, leaving the agent to recover the missing context from long and noisy interaction histories. To bridge this evaluation gap, we introduce \textbf{MemU-Bench}, a mobile benchmark of \textbf{118 vague tasks} grounded in 29 long-horizon user trajectories, where agents must distill raw interaction streams into memory and retrieve the missing context on demand, with task outcomes verified against serialized application databases. Alongside Success Rate, we incorporate Interaction Efficiency (IE) and Success weighted by Path Length (SPL) to expose agents that succeed by exhaustively searching across different applications rather than by \emph{genuinely recalling} the history. Evaluations show that without user history, state-of-the-art systems struggle under such ambiguity: the best-performing Gemini-3.1-Pro achieves an SPL of only 16.4\%, and end-to-end GUI agents fall below 5\%. To address this, we further propose \textbf{AppMem}, a training-free framework that distills user history into a \emph{structured, app-centric} memory bank, tracks interrupted intents across application switches, and selectively retrieves relevant memories via target-app inference. AppMem raises the SPL of GPT-5.5 from 15.7\% to 66.9\% (\textbf{+51.2\%}), outperforms a naive summary baseline by 9.3\%, and delivers consistent gains across evaluated backbones.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2611.00191 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T18:09:53.265Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).