Metraipolis › archive › dc_4651d3585643

Reversible Pseudonymisation of Documents Sent to External LLMs: an Open Bilingual Benchmark and an Engine — arXiv:2610.34012

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.34012
License (as archived)
CC0-1.0
Archived
2026-10-27T08:11:28.924Z
SHA-256 of the archived text
e6d4be2607836931b44c8093a14ee6089d44bb13fda71f81e2a427fc3cb2ac2b
Stable URL of this page
https://metraipolis.com/documento/dc_4651d3585643

How to cite

Reversible Pseudonymisation of Documents Sent to External LLMs: an Open Bilingual Benchmark and an Engine — arXiv:2610.34012 — arXiv (en), https://arxiv.org/abs/2610.34012 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4651d3585643 (Metraipolis archive dc_4651d3585643, sha256 e6d4be2607836931b44c8093a14ee6089d44bb13fda71f81e2a427fc3cb2ac2b)
BibTeX
@misc{dc4651d3585643,
  title = {Reversible Pseudonymisation of Documents Sent to External LLMs: an Open Bilingual Benchmark and an Engine — arXiv:2610.34012},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.34012},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4651d3585643; sha256 e6d4be2607836931b44c8093a14ee6089d44bb13fda71f81e2a427fc3cb2ac2b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.34012. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Professionals send working documents to external large language models. A common safeguard is reversible pseudonymisation: personal data are replaced with placeholders before the request and restored locally in the answer. We present an open benchmark of five existing open tools and a regular-expression baseline on Russian and UK English, and an open engine built on the two strongest tools. The benchmark excludes each tool's own authors' dataset from its verdict, compares the engine with each tool on identical datasets for detection, reports masking precision, and tests on a held-out set of synthetic documents whose files, hashes, decision rules, run plan and stopping rule were published before any run. Two local LLMs test whether placeholders survive. All evaluation data are synthetic, and the benchmark was run by the author of the engine; code, data, predictions and raw answers are released. On external datasets no existing tool meets all predefined thresholds: in UK English the identifier and leak thresholds are not met, in Russian the identifier threshold is borderline. On the last held-out set of 1,000 documents per language, the best existing tools leave a whole value visible in 61% (Russian) and 58% (UK) of documents. On these documents, from the generator it was developed on, the engine does so in 0.3% [0.1; 0.9] and 0.2% [0.1; 0.7], leaves any part of a value visible in 0.8% and 0.2%, and meets all detection thresholds in both languages. In UK English it also restores at least 99% of its placeholders after 760 answers. In Russian it restores only 94.6% [93.6; 95.6] of name placeholders, against a 99% threshold, mainly because the inflection it borrows from pii-guard produces wrong surname forms. The engine was developed over six earlier held-out sets from the same generator; real documents were not tested.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Cryptography and Security (cs.CR).
Pagina del lavoro: https://arxiv.org/abs/2610.34012 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-27T07:56:00.216Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).