Search papers, labs, and topics across Lattice.
This paper systematically evaluates the robustness of three popular machine learning architectures—1D Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM) networks, and Random Forest (RF) ensembles—against adversarial attacks in the context of Network Intrusion Detection Systems (NIDS). Using the ACI-IoT-2023 dataset, the authors subjected each model to FGSM and PGD attacks, revealing that while Random Forest exhibited high baseline accuracy, it suffered a catastrophic drop of 73 percentage points under minimal perturbations, whereas CNN maintained a robust 95.5% accuracy at the same level of attack. These results challenge the assumption that high baseline performance correlates with resilience to adversarial manipulation, providing critical insights for practitioners on model selection in adversarial settings.
Random Forests may boast high accuracy, but they can collapse under minimal adversarial pressure, while CNNs show remarkable resilience.
Network Intrusion Detection Systems (NIDS) heavily utlize Machine Learning (ML) but ML models can be manipulated via adversarial attacks. These attacks add carefully crafted perturbations to network traffic data that leads to misclassifications. While prior work has demonstrated adversarial vulnerabilities in isolated settings, systematic cross-architecture as well as class and category of attack based comparisons under controlled attack conditions remain limited, leaving practitioners without clear guidance on which models to deploy in adversarial environments. This paper asks a simple question: what type of classifier architectures actually hold up when attackers try to manipulate the systems? We put three popular architectures through their paces: a 1D Convolutional Neural Network, a Long Short-Term Memory (LSTM) network, and a Random Forest (RF) ensemble. Using the ACI-IoT-2023 dataset (over 1.2 million samples spanning 12 attack types), we subject each model with FGSM and PGD adversarial attacks, which apply gradient-based perturbations in normalized feature space consistent with established adversarial ML evaluation protocols, at perturbation budgets ranging from $\epsilon=0.01$ to $\epsilon=0.1$. Surprisingly, Random Forest achieved near-perfect baseline accuracy (99.98\%), yet collapsed catastrophically under attack, dropping 73 percentage points at the smallest perturbation we tested. CNN, on the other hand, retained 95.5\% accuracy at $\epsilon=0.01$ and degraded gracefully as perturbations increased. LSTM fell somewhere in between. These findings flip the conventional wisdom where high baseline accuracy means nothing if a model shatters at the first sign of adversarial pressure. For practitioners deploying intrusion detection in adversarial environments, we recommend CNN-based architectures and provide scenario-specific deployment guidance.
