Metraipolis › archive › dc_ffb7aae41669

QuasiLeak-Bench: Evaluating LLM-Based Privacy Detectors on Policy-Conditioned Quasi-Evidence Group Recovery — arXiv:2610.21333

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.21333
License (as archived)
CC0-1.0
Archived
2026-10-16T09:56:02.306Z
SHA-256 of the archived text
4f7ca4e7aad38e06801cafe7ead82f62e43714cf028f443bd25051bcc387b0dc
Stable URL of this page
https://metraipolis.com/documento/dc_ffb7aae41669

How to cite

QuasiLeak-Bench: Evaluating LLM-Based Privacy Detectors on Policy-Conditioned Quasi-Evidence Group Recovery — arXiv:2610.21333 — arXiv (en), https://arxiv.org/abs/2610.21333 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ffb7aae41669 (Metraipolis archive dc_ffb7aae41669, sha256 4f7ca4e7aad38e06801cafe7ead82f62e43714cf028f443bd25051bcc387b0dc)
BibTeX
@misc{dcffb7aae41669,
  title = {QuasiLeak-Bench: Evaluating LLM-Based Privacy Detectors on Policy-Conditioned Quasi-Evidence Group Recovery — arXiv:2610.21333},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.21333},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ffb7aae41669; sha256 4f7ca4e7aad38e06801cafe7ead82f62e43714cf028f443bd25051bcc387b0dc},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.21333. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Natural language messages often contain quasi-identifiers that are weak in isolation but can jointly reveal an individual's private attributes. We call such a jointly revealing set of cues a quasi-evidence group. Existing privacy benchmarks focus on explicit identifiers, attribute inference, or task-driven disclosures, leaving inference from quasi-evidence groups largely unmeasured. In practice, whether exposing such a quasi-evidence group violates privacy also depends on sender-specified disclosure policies. We introduce QuasiLeak-Bench, a benchmark for policy-conditioned quasi-evidence group recovery, which evaluates whether detectors recover complete quasi-evidence groups and flag the disclosures the given policy prohibits. To obtain reliable ground truth at scale, a reverse-construction pipeline first selects weak quasi-identifiers, composes them into quasi-evidence groups verified to jointly reveal a target attribute, and generates documents around this evidence, followed by automated quality control. Experiments show that existing privacy detectors transfer poorly to QuasiLeak-Bench, and locally deployable LLM-based detectors fall far behind proprietary models, mainly because they miss most quasi-evidence groups altogether. However, a 4B model fine-tuned on our training split surpasses the strongest prompted model on held-out documents, showing that the task is learnable.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Cryptography and Security (cs.CR).
Pagina del lavoro: https://arxiv.org/abs/2610.21333 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T08:44:47.640Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).