Metraipolis › archive › dc_49835f348ace

Policy Alignment: New Signals for Membership Auditing in On-Policy Distillation — arXiv:2610.11423

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.11423
License (as archived)
CC0-1.0
Archived
2026-10-09T06:23:19.812Z
SHA-256 of the archived text
7e5483acb14b760870f25ead46800e1616f2066fc09e120bc2a80a095cc6a6e5
Stable URL of this page
https://metraipolis.com/documento/dc_49835f348ace

How to cite

Policy Alignment: New Signals for Membership Auditing in On-Policy Distillation — arXiv:2610.11423 — arXiv (en), https://arxiv.org/abs/2610.11423 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_49835f348ace (Metraipolis archive dc_49835f348ace, sha256 7e5483acb14b760870f25ead46800e1616f2066fc09e120bc2a80a095cc6a6e5)
BibTeX
@misc{dc49835f348ace,
  title = {Policy Alignment: New Signals for Membership Auditing in On-Policy Distillation — arXiv:2610.11423},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.11423},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_49835f348ace; sha256 7e5483acb14b760870f25ead46800e1616f2066fc09e120bc2a80a095cc6a6e5},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.11423. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

On-policy distillation (OPD) trains a student model by aligning its policy with a teacher model on trajectories generated by the student model itself. Through this process, the student policy moves toward the teacher on the prompts used for distillation. However, these prompts are often private and costly, creating a need for prompt-level membership auditing. Existing methods mainly rely on likelihood-based confidence signals or student policy drift between checkpoints, but they do not capture the teacher-induced direction of the student update. In this paper, we propose Policy Alignment Membership Auditing (PAMA), a new auditing framework tailored for OPD. Our key observation is that a member prompt directly contributes to the teacher-guided policy update, while a non-member prompt only experiences indirect effects through cross-prompt generalization. Based on this directional trace, PAMA measures whether the student update moves toward reducing the teacher loss on a candidate prompt. Specifically, we introduce Teacher Alignment Gain (TAG) to estimate the teacher-aligned update direction from model outputs, and further combine it with student drift and uncertainty alignment signals for reliable membership auditing. We evaluate PAMA on six datasets and three teacher-student model families. On MATH, the primary evaluation benchmark, PAMA achieves AUC values of 0.791--0.941, improving AUC by 14.6--20.6% over state-of-the-art baselines.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.11423 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T05:50:31.394Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).