Metraipolis › archive › dc_932571daa674

DegreeSpar: Structured Degree Sparsity for Efficient Secure Transformer Inference — arXiv:2609.32204

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.32204
License (as archived)
CC0-1.0
Archived
2026-10-07T22:34:33.709Z
SHA-256 of the archived text
5bf278e727fdbda854bc0bfb2df79873fb6be3e1082831107d898ee269e82f80
Stable URL of this page
https://metraipolis.com/documento/dc_932571daa674

How to cite

DegreeSpar: Structured Degree Sparsity for Efficient Secure Transformer Inference — arXiv:2609.32204 — arXiv (en), https://arxiv.org/abs/2609.32204 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_932571daa674 (Metraipolis archive dc_932571daa674, sha256 5bf278e727fdbda854bc0bfb2df79873fb6be3e1082831107d898ee269e82f80)
BibTeX
@misc{dc932571daa674,
  title = {DegreeSpar: Structured Degree Sparsity for Efficient Secure Transformer Inference — arXiv:2609.32204},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.32204},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_932571daa674; sha256 5bf278e727fdbda854bc0bfb2df79873fb6be3e1082831107d898ee269e82f80},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.32204. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Secure Transformer inference protects sensitive inputs but incurs substantial cryptographic overhead, with nonlinear operations such as Softmax and GeLU becoming major bottlenecks. Existing compression methods reduce nonlinear complexity, sequence-dependent computation, or model structure through separately defined compression variables. Under aggressive compression, however, these independently optimized perturbations can accumulate: at a matched compression level, stacking representative approximation, token-pruning, and model-pruning methods reduces ViT-S accuracy from 80.20% to 76.41%. We introduce DegreeSpar, which formulates secure Transformer compression as structured sparsification over nonlinear polynomial degrees. Polynomial degree directly controls the cost of secure nonlinear evaluation, while computation-aligned zero-degree structures expose token-level and model-dimension computation as removable within the same optimization space. DegreeSpar further incorporates approximation-aware training for low-degree Softmax and GeLU, enabling aggressive degree reduction and creating the optimization headroom required for structured computation removal. Across vision and language Transformers, DegreeSpar consistently improves the accuracy-latency trade-off across model scales, tasks, and sequence lengths, achieving speedups from 2.29x to 6.63x over the corresponding baselines. Under the same network setting, DegreeSpar achieves 92.68% accuracy on BERT/SST-2 in 110.55 s, compared with 92.66% in 167.26 s for CipherPrune, the closest prior hybrid secure-inference approach. These results establish structured polynomial degree as an effective shared optimization space for secure Transformer compression.

Tema degli esploratori: fiducia digitale e crittografia (retro-catalogo arXiv: annunci del 2026-09-29 in cs.CR).
Categorie arXiv: Cryptography and Security (cs.CR) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.32204 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T22:13:03.665Z dalla pagina https://arxiv.org/catchup/cs.CR/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).