arXiv AI

A Three-Axis Stress Test of LLM vs Classical ML for Network Intrusion Detection under Distribution Shift and Adversarial Evasion

The study compares XGBoost and RoBERTa‑LoRA for network intrusion detection across three evaluation axes: same‑dataset performance, cross‑dataset transfer, and adversarial evasion. Both models perform similarly on the same dataset, but XGBoost outperforms RoBERTa‑LoRA by 15 F1 points and 25 balanced accuracy points when transferred to a different network, while RoBERTa‑LoRA wins by about 17 F1 points under adversarial evasion. Feature‑leakage ablation shows that cross‑dataset transfer improvements are non‑monotonic and directional, suggesting leakage is spread across features rather than isolated. "whyItMatters":"The findings demonstrate that a model’s superiority depends on the specific robustness axis evaluated, underscoring the need for multi‑axis, multi‑metric testing in network intrusion detection research."

arXiv Machine Learning
Jun 10

Do Transformers Actually Help Intrusion Detection? A Temporal Sequence Evaluation on CIC-IDS2017

arXiv:2606. 11098v1 Announce Type: cross Abstract: Recent deep learning approaches for network intrusion detection increasingly incorporate temporal architectures such as recurrent networks and Transformers, often reporting near-perfect performance on CIC-IDS2017.

By Zach Moczkodan (Royal Military College of Canada, Kingston, Canada), Hany Ragab (Royal Military College of Canada, Kingston, Canada)
arXiv AI
Jun 9

SHIELD-IDS: Structurally Heterogeneous Ensemble with Integrated Layered Defense for Intrusion Detection Systems

arXiv:2606. 07716v1 Announce Type: cross Abstract: Adversarial attacks pose a serious and growing threat to Machine Learning (ML)-based Intrusion Detection Systems (IDS), where imperceptible perturbations to network flow features can systematically mislead classifiers into accepting malicious traffic as benign.

By Maryam Zaman, Muhammad Khuram Shahzad