Metraipolis › archive › dc_f2ba7582a7be

ProductWebBench: Evaluating Website Continuity Beyond Task Completion — arXiv:2610.31118

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.31118
License (as archived)
CC0-1.0
Archived
2026-10-25T08:34:50.754Z
SHA-256 of the archived text
db75b212f23b210548158ece20c4e37c1260b718fed92825a814f7b496ece94f
Stable URL of this page
https://metraipolis.com/documento/dc_f2ba7582a7be

How to cite

ProductWebBench: Evaluating Website Continuity Beyond Task Completion — arXiv:2610.31118 — arXiv (en), https://arxiv.org/abs/2610.31118 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f2ba7582a7be (Metraipolis archive dc_f2ba7582a7be, sha256 db75b212f23b210548158ece20c4e37c1260b718fed92825a814f7b496ece94f)
BibTeX
@misc{dcf2ba7582a7be,
  title = {ProductWebBench: Evaluating Website Continuity Beyond Task Completion — arXiv:2610.31118},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.31118},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f2ba7582a7be; sha256 db75b212f23b210548158ece20c4e37c1260b718fed92825a814f7b496ece94f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.31118. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Frontend coding agents are usually graded on one question: did the requested target get built. Real engineering changes carry a second obligation, because the rest of the site has to keep working. We call that second obligation website continuity, and introduce ProductWebBench, a benchmark that scores both on living frontend repositories: 320 Change tasks over existing code and 80 long-horizon Build tasks. Every task freezes a repository snapshot, staged requirements, browser evidence and deterministic checks before evaluation, and enters only once a reference passes every check and plausible-wrong ones fail. A simulated user reveals the requirements one stage at a time, and every check cleared earlier stays armed for the rest of the run. Across 8,737 runs from 13 models the two come apart: of the Change runs passing every requested-work check, 28.2% still fail a continuity check -- a blank route, horizontal overflow, console errors -- and 10.8% push a browser state they had already passed back to failing. Because the same checks also run on the untouched repository, each regression carries exact provenance, not a post-hoc label: on the 586 runs where both readings exist, 18.3% are caught breaking a verified state. Interaction is a further blind spot, failing in 29.5% of runs whose static DOM already passes. ProductWebBench therefore scores a patch against the living system it lands in, not the target alone.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Software Engineering (cs.SE) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.31118 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-25T08:31:13.224Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).