Federated Attention Autoencoders with a Stochastic Aggregation Scheme for Anomaly Detection

2026-08-09

AI Summary

This work addresses the challenge of anomaly detection in decentralized data environments where data cannot be shared across clients. To tackle this issue, the authors propose a federated learning approach that integrates attention mechanisms with autoencoders. The key innovation lies in the design of two novel stochastic aggregation functions specifically tailored for attention-based autoencoders, which effectively preserve critical information from local memory modules on client devices and enhance the global model’s capacity for information integration. Experimental results on the KDDCUP99 dataset demonstrate that the proposed method outperforms conventional autoencoders, achieving relative improvements of 2.9% in F1 score and 5.1% in AUC-ROC, thereby significantly advancing anomaly detection performance in federated settings.

Abstract

Outlier detection in decentralized data environments is a challenging task for many machine learning implementations, particularly in settings where data cannot be shared. Recently, there have been advances in federated outlier detection, some of which are based on the use of autoencoder networks. The introduction of attention mechanisms to autoencoders boosts their efficiency. However, the application of attention-based models in federated learning remains underdeveloped due to the absence of proper aggregation functions for these types of networks. In our work, we propose two novel aggregation functions tailored for attention-based autoencoders, which better preserve the learned information stored within the memory modules of these networks. We evaluated our approach on the KDDCUP10 dataset, and we showed that the proposed methods achieve up to 2.9\% and 5.1\% better results for F1 score and AUC ROC respectively when compared to traditional autoencoders.

Problem

• federated learning
• anomaly detection
• attention mechanism
• autoencoder
• stochastic aggregation

Innovation

• Federated Learning
• Attention Mechanism
• Autoencoder
• Stochastic Aggregation
• Anomaly Detection