Metraipolis › archive › dc_f9a7bee1bee6

Why Ghost Outputs Teach: A Kernel-Based Understanding of Subliminal Learning — arXiv:2609.23260

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.23260
License (as archived)
CC0-1.0
Archived
2026-10-09T07:07:07.247Z
SHA-256 of the archived text
1352b584c6cca838c1c726cda0274ff1d36fd4abf9a186902a61ad02f4e79e46
Stable URL of this page
https://metraipolis.com/documento/dc_f9a7bee1bee6

How to cite

Why Ghost Outputs Teach: A Kernel-Based Understanding of Subliminal Learning — arXiv:2609.23260 — arXiv (en), https://arxiv.org/abs/2609.23260 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f9a7bee1bee6 (Metraipolis archive dc_f9a7bee1bee6, sha256 1352b584c6cca838c1c726cda0274ff1d36fd4abf9a186902a61ad02f4e79e46)
BibTeX
@misc{dcf9a7bee1bee6,
  title = {Why Ghost Outputs Teach: A Kernel-Based Understanding of Subliminal Learning — arXiv:2609.23260},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.23260},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f9a7bee1bee6; sha256 1352b584c6cca838c1c726cda0274ff1d36fd4abf9a186902a61ad02f4e79e46},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.23260. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Subliminal Learning (SL) is a recently identified phenomenon in which a student model acquires downstream task capabilities by matching seemingly unrelated auxiliary outputs from a teacher, despite never observing task labels, task-specific outputs, or the original training data. While recent studies have identified where subliminal signals may reside, the optimization mechanism underlying this phenomenon remains poorly understood. In this work, we provide a mechanistic understanding of SL through the lens of learning dynamics. Specifically, we derive a chained cross-task kernel that explicitly links ghost-output supervision to changes in task predictions through shared backbone representations. Our unified analytical framework provides a rigorous mathematical explanation for three central empirical puzzles in SL: (i) under shared initialization, the transfer operator forms a strictly Positive Semi-Definite (PSD) structure, guaranteeing that ghost-output optimization aligns the student with the teacher's true task objective without explicit label exposure; (ii) the ghost-output dimensionality acts as an explicit rank bottleneck governing the transfer of task-relevant features; and (iii) synthetic, high-entropy inputs function as broadband probes that maximize cross-task kernel overlap, explaining why random noise consistently outperforms structured data for subliminal transfer. Experiments on the canonical ghost-output setting validate all three theoretical predictions, providing the first learning-dynamics-based theoretical explanation of how ghost-output supervision gives rise to subliminal learning.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-22 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.23260 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T06:58:09.282Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).