Comparative Evaluation of VAEs, VAE-GANs and AAEs for Anomaly Detection in Network Intrusion Data

Abstract

With cyberattacks growing in frequency and sophistication, effective anomaly detection is critical for securing networks and systems. This study provides a comparative evaluation of deep generative models for detecting anomalies in network intrusion data. The key objective is to determine the most accurate model architecture. Variational autoencoders (VAEs), VAE-GANs, and adversarial autoencoders (AAEs) are tested on the NSL-KDD dataset containing normal traffic and different attack types. Results show that AAEs significantly outperform VAEs and VAE-GANs, achieving AUC scores up to 0.96 and F1 scores of 0.76 on novel attacks. The adversarial regularization of AAEs enables superior generalization capabilities compared to standard VAEs. VAE-GANs exhibit better accuracy than VAEs, demonstrating the benefits of adversarial training. However, VAE-GANs have higher computational requirements. The findings provide strong evidence that AAEs are the most effective deep anomaly detection technique for intrusion detection systems. This study delivers novel insights into optimizing deep learning architectures for cyber defense. The comparative evaluation methodology and results will aid researchers and practitioners in selecting appropriate models for operational network security.