Metraipolis › archive › dc_4a2d1edd5b9e

Context-dependent agent evaluation with orthogonal equilibrium learning — arXiv:2609.31897

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.31897
License (as archived)
CC0-1.0
Archived
2026-10-07T20:54:58.259Z
SHA-256 of the archived text
15805719fc0f6d53ac8f2b7d67e406e4c03aa7bbe25d51c7003535ef4cff8d27
Stable URL of this page
https://metraipolis.com/documento/dc_4a2d1edd5b9e

How to cite

Context-dependent agent evaluation with orthogonal equilibrium learning — arXiv:2609.31897 — arXiv (en), https://arxiv.org/abs/2609.31897 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4a2d1edd5b9e (Metraipolis archive dc_4a2d1edd5b9e, sha256 15805719fc0f6d53ac8f2b7d67e406e4c03aa7bbe25d51c7003535ef4cff8d27)
BibTeX
@misc{dc4a2d1edd5b9e,
  title = {Context-dependent agent evaluation with orthogonal equilibrium learning — arXiv:2609.31897},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.31897},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4a2d1edd5b9e; sha256 15805719fc0f6d53ac8f2b7d67e406e4c03aa7bbe25d51c7003535ef4cff8d27},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.31897. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Many applications require to evaluate agents under contextual information (e.g., a prompt, task, or user group). We study how to perform such context-dependent agent evaluation from offline feedback. Existing score-based models for this purpose (e.g., Bradley-Terry) impose a transitive preference ordering, which fails to reflect collective preferences when human judgements are heterogeneous. Inspired by social choice theory, we frame evaluation as a contextual game between two players, each selecting a distribution over agents as the strategy to receive greater collective preference than the other. Then, the support of the Nash equilibrium defines a context-specific set of winners. However, learning context-specific equilibria from offline logs is difficult because each context reveals human feedback on only a subset of agents, and, hence, a naive plug-in estimator can therefore be biased. To address these challenges, we propose NashEval, a general framework for robust contextual equilibrium learning. NashEval first constructs debiased estimates of the contextual payoff matrix that characterizes the game. NashEval then learns the context-to-equilibrium mapping with a tailored orthogonal loss, which avoids the need to solve a separate game for each context. We show theoretically that errors in estimating the nuisance functions underlying the payoff matrix affect the risk of the learned equilibrium (i.e., exploitability) only through higher-order terms. Across various experiments, NashEval improves robustness of equilibrium learning and consistently identifies the set of top-performing agents across contexts.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.31897 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T20:29:00.659Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).