Metraipolis › archive › dc_b476f919082f

When Agents Collude: Weak-to-Strong On-Policy Distillation for Mitigating Strategic Deception in Multi-Agent Systems — arXiv:2610.33011

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.33011
License (as archived): CC0-1.0
Archived: 2026-10-27T08:51:31.585Z
SHA-256 of the archived text: 9eee0ba7b19146b074e7e64497dafeb9b7043cfac1be7af5525071925505da2a
Stable URL of this page: https://metraipolis.com/documento/dc_b476f919082f

How to cite

When Agents Collude: Weak-to-Strong On-Policy Distillation for Mitigating Strategic Deception in Multi-Agent Systems — arXiv:2610.33011 — arXiv (en), https://arxiv.org/abs/2610.33011 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b476f919082f (Metraipolis archive dc_b476f919082f, sha256 9eee0ba7b19146b074e7e64497dafeb9b7043cfac1be7af5525071925505da2a)

BibTeX

@misc{dcb476f919082f,
  title = {When Agents Collude: Weak-to-Strong On-Policy Distillation for Mitigating Strategic Deception in Multi-Agent Systems — arXiv:2610.33011},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.33011},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b476f919082f; sha256 9eee0ba7b19146b074e7e64497dafeb9b7043cfac1be7af5525071925505da2a},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.33011. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

While recent advances in autonomous agents enable them to coordinate on increasingly complex tasks, prior work shows that this growing capacity for coordination can create safety risks, including collusion, in which agents jointly pursue misaligned actions while concealing the true basis of those actions from oversight. We study this risk in a simulated insider-trading environment, comparing single agents with sequential and decentralized teams across seven language models from three families. By varying agents' system-level objectives and pressure conditions, we examine when coordinated agents act on material nonpublic information, conceal the basis of a misaligned trade in a report, and maintain that deception under follow-up questioning. Compared to single-agent decision-making, multi-agent interaction raises misaligned-action rates from 35.4% to 68.2% and persistent-deception rates from 31.0% to 62.2%, both measured over all runs. These effects are largest under existential pressure, where teams sustain deception through follow-up questioning in 87% of all runs. Qualitative analysis reveals shared cover stories and coordinated rationales, suggesting group-level strategic deception rather than isolated reporting errors. To mitigate this behavior, we introduce PACT, a weak-to-strong on-policy distillation framework for mitigating collusive deception. Using generic honesty prompts, PACT trains a stronger student on its own sampled continuations, matching its next-token distribution to that of a smaller, frozen honesty-aligned teacher via reverse-KL supervision. Under existential pressure with ethical instructions, PACT reduces multi-agent persistent deception from 80% to 3%. Our results show that multi-agent coordination amplifies strategic deception and that on-policy honesty alignment can substantially mitigate these failures.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.MA, nuovi lavori del giorno).
Categorie arXiv: Multiagent Systems (cs.MA) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.33011 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-27T08:51:13.276Z dalla pagina https://arxiv.org/list/cs.MA/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).