Metraipolis › archive › dc_3cfe73093cac

An Open-Source End-to-End FHE Implementation for Privacy-Preserving Llama 3 8B Inference — arXiv:2609.12378

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.12378
License (as archived)
CC0-1.0
Archived
2026-10-16T03:14:29.342Z
SHA-256 of the archived text
2efc509ded800bd8549f348ff8ea9a7f1453e601ce53319b7d5108941193dd07
Stable URL of this page
https://metraipolis.com/documento/dc_3cfe73093cac

How to cite

An Open-Source End-to-End FHE Implementation for Privacy-Preserving Llama 3 8B Inference — arXiv:2609.12378 — arXiv (en), https://arxiv.org/abs/2609.12378 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3cfe73093cac (Metraipolis archive dc_3cfe73093cac, sha256 2efc509ded800bd8549f348ff8ea9a7f1453e601ce53319b7d5108941193dd07)
BibTeX
@misc{dc3cfe73093cac,
  title = {An Open-Source End-to-End FHE Implementation for Privacy-Preserving Llama 3 8B Inference — arXiv:2609.12378},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.12378},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3cfe73093cac; sha256 2efc509ded800bd8549f348ff8ea9a7f1453e601ce53319b7d5108941193dd07},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.12378. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Cloud LLM services typically require users to send prompts to a model provider, creating a privacy risk. Fully homomorphic encryption (FHE) lets a server perform inference without decrypting the input, but representing data as ciphertexts adds storage and computational overhead. In CKKS-based LLM inference, the packing scheme maps logical tensors to ciphertexts and slots. It therefore determines the ciphertext count and the homomorphic cost of linear layers, and it constrains how data pass between linear layers, attention, and nonlinear computation. As models and sequences grow, inefficient layouts accumulate encoding, compute, and layout-conversion overhead. We present Odin, an FHE inference system that co-designs ciphertext packing and model execution for Llama. Starting from a THOR-style baseline whose bottleneck is weight encoding, Odin uses a feature-major cross-layer layout to unify residual connections and layer interfaces, and builds transient intra-operator layouts for linear projections and attention. This reduces redundant plaintext encoding of weights in wide projections. Within attention, QK^T produces scores that Softmax can consume directly, and PV consumes the resulting probabilities, avoiding intermediate repacking. For nonlinear ops, we use minimax polynomial approximation with input-range control and joint error allocation guided by model quality, reducing polynomial degree and multiplicative depth. To our knowledge, Odin is the first open-source end-to-end GPU CKKS implementation of Llama-3. With Llama-3-8B weights and a 128-token input, Odin evaluates all 32 Transformer layers on a single NVIDIA H100 80 GB GPU. Server-side end-to-end FHE evaluation takes 366.4 s and 58.9 GiB peak device memory. Under the same model, input, CKKS parameters, and hardware, THOR takes 1651.9 s, a 4.51x speedup.

Tema degli esploratori: fiducia digitale e crittografia (retro-catalogo arXiv: annunci del 2026-09-14 in cs.CR).
Categorie arXiv: Cryptography and Security (cs.CR).
Pagina del lavoro: https://arxiv.org/abs/2609.12378 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T03:05:08.055Z dalla pagina https://arxiv.org/catchup/cs.CR/2026-09-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).