Metraipolis › archive › dc_d8dca6fa281b

PII-TRACE: A Benchmark for Context-Aware PII Detection in Multi-Turn LLM Conversations — arXiv:2609.22200

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.22200
License (as archived): CC0-1.0
Archived: 2026-10-09T06:49:14.341Z
SHA-256 of the archived text: 247e522df2bb719ac9553664a0202e87dc531726a449765c59435534a924f851
Stable URL of this page: https://metraipolis.com/documento/dc_d8dca6fa281b

How to cite

PII-TRACE: A Benchmark for Context-Aware PII Detection in Multi-Turn LLM Conversations — arXiv:2609.22200 — arXiv (en), https://arxiv.org/abs/2609.22200 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d8dca6fa281b (Metraipolis archive dc_d8dca6fa281b, sha256 247e522df2bb719ac9553664a0202e87dc531726a449765c59435534a924f851)

BibTeX

@misc{dcd8dca6fa281b,
  title = {PII-TRACE: A Benchmark for Context-Aware PII Detection in Multi-Turn LLM Conversations — arXiv:2609.22200},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.22200},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d8dca6fa281b; sha256 247e522df2bb719ac9553664a0202e87dc531726a449765c59435534a924f851},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.22200. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

LLM assistants and agentic systems log long multi-turn conversations. AI providers often scan these conversations for Personally Identifiable Information (PII) and mask the PII before storing or processing conversation data. Yet most PII detectors and benchmarks target self-contained records rather than cross-turn evaluation. To evaluate PII detection across turns in multi-turn conversations, we introduce PII-TRACE (Tracing Recurring PII Across Conversational Exchanges), to our knowledge the first PII benchmark to assess whether detectors identify PII in conversational contexts and cover every mention of a recurring identifier across turns. PII-TRACE contains 13,148 synthetic multi-turn dialogues in 13 languages with character-level spans and identifier clusters. Across eleven baselines, including frontier LLMs, no detector achieves full entity-level coverage without substantial false positives on PII-free conversations, and single-pass reading loses a third of the gold characters on long dialogues. To close this gap, we introduce PII-Tracer, a compact 0.6B-parameter detector trained with conversation-level supervision. PII-Tracer attains the highest entity-level coverage of any system we evaluate and also performs strongly on standard single-record benchmarks.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-22 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.22200 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T06:48:44.871Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).