Metraipolis › archive › dc_15d52d2518db
From Trajectories to Grounded Preferences: Process Preference Synthesis via Interaction Element Graphs for Web PRMs — arXiv:2609.32351
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.32351
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T20:55:20.727Z
- SHA-256 of the archived text
c9dff8da8e0b1788cb5042a45d3e9f950035db2489d8475ebe6ff4b43f71f0f5- Stable URL of this page
https://metraipolis.com/documento/dc_15d52d2518db
How to cite
From Trajectories to Grounded Preferences: Process Preference Synthesis via Interaction Element Graphs for Web PRMs — arXiv:2609.32351 — arXiv (en), https://arxiv.org/abs/2609.32351 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_15d52d2518db (Metraipolis archive dc_15d52d2518db, sha256 c9dff8da8e0b1788cb5042a45d3e9f950035db2489d8475ebe6ff4b43f71f0f5)
BibTeX
@misc{dc15d52d2518db,
title = {From Trajectories to Grounded Preferences: Process Preference Synthesis via Interaction Element Graphs for Web PRMs — arXiv:2609.32351},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.32351},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_15d52d2518db; sha256 c9dff8da8e0b1788cb5042a45d3e9f950035db2489d8475ebe6ff4b43f71f0f5},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.32351. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Comparative Process Reward Models (PRMs) provide critical step-level guidance for autonomous web agents by evaluating state-conditioned preferences between candidate actions. However, existing preference training data synthesized via multi-policy sampling suffers from a severe scarcity of Grounded Minimal Contrastive Pairs (GMCPs)-where competing candidates target genuine on-page elements with identical action types. In representative baselines preference data (namely, WebArbiter), GMCPs account for merely 24.19%, biasing PRMs during training to rely on shallow shortcuts (such as element hallucinations and action type mismatches) rather than acquiring genuine contextual decision semantics. To address these challenges, we propose SURFPRM, a graph-guided process preference synthesis framework for comparative Web PRMs. SURFPRM structures web demonstrations into a persistent Interaction Element Graph that acts as an environment-grounded negative action proposal mechanism, systematically synthesizing contrastive negative actions across spatial, temporal, and spatiotemporal confusion axes. This elevates the GMCP proportion from 24.19% to 74.60%, producing the curated SURFPRM-DATA dataset. Across six open-source backbones (3B to 9B parameters), PRMs trained on SURFPRM-DATA outperform baseline-trained models on average on WEBPRMBENCH and rival leading proprietary LLMs. In downstream reward-guided trajectory search on WEBARENA-LITE, SURFPRM provides step-level guidance for both GPT-4o (+14.21%) and GPT-4o-mini (+12.83%) policies, yielding substantial improvements in complex web task success rates. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.32351 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T20:29:00.762Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).