Metraipolis › archive › dc_8af70577e76a

Coverage, Not Difficulty, Sets How Much Synthetic Data an Activation Probe Needs — arXiv:2610.10594

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.10594
License (as archived)
CC0-1.0
Archived
2026-10-09T06:20:50.223Z
SHA-256 of the archived text
aa88137f82c38ae5e8403ae6544d10f1df1a0871d7aa9fd7392626c5a6d12af9
Stable URL of this page
https://metraipolis.com/documento/dc_8af70577e76a

How to cite

Coverage, Not Difficulty, Sets How Much Synthetic Data an Activation Probe Needs — arXiv:2610.10594 — arXiv (en), https://arxiv.org/abs/2610.10594 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8af70577e76a (Metraipolis archive dc_8af70577e76a, sha256 aa88137f82c38ae5e8403ae6544d10f1df1a0871d7aa9fd7392626c5a6d12af9)
BibTeX
@misc{dc8af70577e76a,
  title = {Coverage, Not Difficulty, Sets How Much Synthetic Data an Activation Probe Needs — arXiv:2610.10594},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.10594},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8af70577e76a; sha256 aa88137f82c38ae5e8403ae6544d10f1df1a0871d7aa9fd7392626c5a6d12af9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.10594. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Activation probes that monitor deployed language models are trained on synthetic conversations, and how many a probe needs is open. We trace learning curves over 10-590 synthetic samples for three monitoring concepts, high-stakes situations, replies harmful to a person, and replies that do not follow the user's instruction, on fourteen held-out evaluation distributions and four probe models, varying the generator LLM and the prompt's detail. The need is set by what is monitored: probes for high-stakes and harmful are within a few hundredths of their plateau from 80 samples on Gemma-3-27B-IT, instruction probes need several times as many, and the ordering holds on three smaller probe models and on real samples (from dev set). Prior work advises spending a generation budget on breadth, more kinds of data, over depth, more of each kind. We read the depth a concept needs as the half-gain size of a fitted curve, the number of samples at which half the gain is in hand. Concept and distribution account for 42-45% of its variance, the generator, probe model, and prompt detail for under 10%. What sets the value of the half-gain size is coverage, not per-kind difficulty: the number of samples of its own kind a distribution needs to saturate. Every kind, one per evaluation distribution, has a median half-gain size of 7-11 own-kind synthetic samples under all three concepts alike. What differs is how far samples of one kind transfer to the concept's other kinds, almost fully under high-stakes, less under harmful, and least under instruction, which accounts for most of the gap between concepts on generated and real samples. Breadth therefore pays differently by concept: many kinds are necessary under instruction, where no kind covers another, and nearly redundant under high-stakes, where one kind covers the rest. We release the evaluation suites, dev sets, and generated sets.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.10594 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T05:50:30.813Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).