Metraipolis › archive › dc_a7d8adfaeb99

JevAdvBench: A Benchmark and Black-Box Attacks for Reinforcement Learning for Calibrated Decisions Models — arXiv:2609.31142

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.31142
License (as archived)
CC0-1.0
Archived
2026-10-08T01:39:43.198Z
SHA-256 of the archived text
6d57573f09c6b41d581df01f37267d4f6c019b7c826d5d2f7ecc91bfe91a1c47
Stable URL of this page
https://metraipolis.com/documento/dc_a7d8adfaeb99

How to cite

JevAdvBench: A Benchmark and Black-Box Attacks for Reinforcement Learning for Calibrated Decisions Models — arXiv:2609.31142 — arXiv (en), https://arxiv.org/abs/2609.31142 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a7d8adfaeb99 (Metraipolis archive dc_a7d8adfaeb99, sha256 6d57573f09c6b41d581df01f37267d4f6c019b7c826d5d2f7ecc91bfe91a1c47)
BibTeX
@misc{dca7d8adfaeb99,
  title = {JevAdvBench: A Benchmark and Black-Box Attacks for Reinforcement Learning for Calibrated Decisions Models — arXiv:2609.31142},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.31142},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a7d8adfaeb99; sha256 6d57573f09c6b41d581df01f37267d4f6c019b7c826d5d2f7ecc91bfe91a1c47},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.31142. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Models trained with reinforcement learning for calibrated decisions (RLCD), such as Jev, answer a typed question about an input, the state, with a probability, a choice, or a score, and software acts on the answer without a person reading it. Their robustness has not been measured: adversarial benchmarks score what a model generates or executes, whereas a typed model generates nothing and returns a well-formed answer even when manipulated. Measurement is also hard, because identical requests can return different answers, most available labels come from the model itself, and the API preprocesses each request out of view. Our key idea is to score each attacked decision against the model's own clean decision rather than against labels, and to read it against the change caused by an identical re-run. Building on this, we introduce JevAdvBench, to our knowledge the first adversarial benchmark for RLCD models, with 812 typed questions over 66 scenarios, and a black-box attack suite of 9,744 single-edit variants that each edit one part of a request, with billed input tokens confirming that the edit reached the model. On jev-1.13.0, rewording stays within 1.2 percentage points of the re-run baseline, and fields outside the schema never reach the model. In contrast, one unverified opinion appended to the state flips 12.1% of decisions, statistically tied with the strongest injected command (10.1%), and pushes 38% of confident answers below the 0.8 confidence threshold that routes them to human review. Applications built on RLCD models should therefore treat the state as untrusted, argued input. Project website: this https URL

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-28 in cs.AI).
Categorie arXiv: Cryptography and Security (cs.CR) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.31142 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T01:12:14.538Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).