Metraipolis › archive › dc_34aeec4a27ea
Attention, Anomalies! Handling Attention Layers in Unsupervised Federated Outlier Detection — arXiv:2608.04753
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.04753
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-31T11:35:25.225Z
- SHA-256 of the archived text
13412c690cd096b9b48958238f27e60806706357e7bbdb5af89a075711172258- Stable URL of this page
https://metraipolis.com/documento/dc_34aeec4a27ea
How to cite
Attention, Anomalies! Handling Attention Layers in Unsupervised Federated Outlier Detection — arXiv:2608.04753 — arXiv (en), https://arxiv.org/abs/2608.04753 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_34aeec4a27ea (Metraipolis archive dc_34aeec4a27ea, sha256 13412c690cd096b9b48958238f27e60806706357e7bbdb5af89a075711172258)
BibTeX
@misc{dc34aeec4a27ea,
title = {Attention, Anomalies! Handling Attention Layers in Unsupervised Federated Outlier Detection — arXiv:2608.04753},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.04753},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_34aeec4a27ea; sha256 13412c690cd096b9b48958238f27e60806706357e7bbdb5af89a075711172258},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.04753. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Attention layers are the backbone of today's most powerful and impactful models. Models with multi-million and billion parameters rely on contextual knowledge provided by attention layers. However, their use goes well beyond just being the core component of large language models. One particularly interesting application is in Memory Augmented Autoencoders (MemAE), specifically for unsupervised representation learning in outlier detection tasks. It was shown that attention helps these models be more effective in centralized learning scenarios. Our work aims to address the lack of specialized aggregation techniques in Federated Learning (FL) when it comes to MemAE models. In this paper we analyze the intricacies of the architecture behind Memory Augmented Autoencoders, and propose novel, guided approaches to effectively aggregate these models in federated scenarios. We demonstrate our approach on non-IID datasets and show that these novel aggregation schemes are more robust when dealing with numerous edge nodes in environments with unbalanced datasets, specifically for unsupervised anomaly detection scenarios. This approach improves the performance of even very shallow autoencoders, allowing them to be used in resource constrained environments. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-06 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2608.04753 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-31T11:24:48.795Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-06?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).