Metraipolis › archive › dc_43be8d7d7155

Double Blind Evals: Resolving the Dual Confidentiality Dilemma in AI Safety Auditing — arXiv:2611.12604

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.12604
License (as archived)
CC0-1.0
Archived
2026-11-10T18:44:12.786Z
SHA-256 of the archived text
f0b988a2b83ebd3a4a1ac5df9719c3b1c96cac90e7e67f7d384844e938a52e6a
Stable URL of this page
https://metraipolis.com/documento/dc_43be8d7d7155

How to cite

Double Blind Evals: Resolving the Dual Confidentiality Dilemma in AI Safety Auditing — arXiv:2611.12604 — arXiv (en), https://arxiv.org/abs/2611.12604 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_43be8d7d7155 (Metraipolis archive dc_43be8d7d7155, sha256 f0b988a2b83ebd3a4a1ac5df9719c3b1c96cac90e7e67f7d384844e938a52e6a)
BibTeX
@misc{dc43be8d7d7155,
  title = {Double Blind Evals: Resolving the Dual Confidentiality Dilemma in AI Safety Auditing — arXiv:2611.12604},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.12604},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_43be8d7d7155; sha256 f0b988a2b83ebd3a4a1ac5df9719c3b1c96cac90e7e67f7d384844e938a52e6a},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.12604. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

AI safety evaluations often involve sensitive prompts and proprietary models. Evaluators want to keep their evaluation prompts secret from model owners to prevent purposeful or incidental data leakage, ensuring longer usefulness of their benchmark. Model owners in turn wish to keep their proprietary model weights and inference code within their own secure infrastructure to protect their IP. We present a way forward for "double-blind evals", which provide cryptographic guarantees that an evaluator's prompts are not shared with the model owner, while simultaneously maintaining the confidentiality of the model weights and inference code. To accomplish this, we use secure enclaves that encrypt data in RAM at the hardware level. In a world-first pilot, we test this approach by evaluating a proprietary Google DeepMind model (Gemini 2.5 Flash Lite) against a selection of the private AILuminate benchmark corpus, using an NVIDIA H100 secure enclave on Google Cloud Platform, and using OpenMined's PySyft to maintain data and model privacy. We believe this effort marks the first use of this technology to provide mutual secrecy for one organization's frontier-class proprietary model weights and another organization's closed benchmark.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Cryptography and Security (cs.CR) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.12604 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-10T18:37:37.700Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).