arXiv AI

On the Reliability and Stability of Selective Methods in Malware Classification Tasks

The paper examines whether modern drift‑adaptive malware classifiers provide reliable and stable confidence estimates when faced with distribution shifts. It introduces Aurora, a framework that evaluates confidence quality and operational resilience, and presents metrics that extend beyond point‑in‑time performance to assess stability over time. The study finds that state‑of‑the‑art models exhibit fragility across datasets with varying drift severity, indicating a need to revisit underlying assumptions.

arXiv Machine Learning
Jul 28

EXE-Bench: Ranking the Tradeoffs of AI-based Windows Malware Detectors for Real-World Usability

arXiv:2607. 24177v1 Announce Type: cross Abstract: Due to the lack of systematic evaluations, we are not yet able to determine which AI-based Windows malware detector to deploy in production, since existing evaluations (i) differ in terms of data used for both training and testing; (ii) do not consider temporal analysis to showcase whether models withstand the passage of time; (iii) avoid security evaluations with adversarial attacks that could highlight their brittleness against content-injection attacks; and (iv) neglect the computational requirements for deployment, risking slow inference on endpoints.

By Andrea Ponte, Daniel Gibert, Matous Kozak, Dmitrijs Trizna, Maura Pintor, Battista Biggio, Fabio Roli, Luca Demetrio
Hugging Face Trending Papers
Jul 2

Beyond Gradient-Based Attacks: Adversarial Robustness and Explainability Stability in Cybersecurity Classifiers

Adversarial attacks on cybersecurity classifiers pose a dual threat: degrading predictions and destabilising the SHAP-based explanations that security analysts rely on to understand and triage alerts. We extend our prior MLP conference study to Random Forest and XGBoost across four tabular security datasets (phishing URLs, UNSW-NB15, NF-ToN-IoT, HIKARI-2021), evaluating five attacks including three black-box methods applicable to non-differentiable tree models.

arXiv AI
Aug 25

Adapter-Based Few-Shot Continual Learning for Malicious Packet Recognition

The paper addresses the challenge of adapting malware detection systems to new threats without retraining from scratch, focusing on the Few-Shot Class-Incremental Learning (FSCIL) setting. It proposes a hybrid framework that uses a self-supervised learning backbone pre-trained on malware packets, incorporates Low-Rank Adaptation (LoRA) to adapt the model while preserving core representations, and employs a prototype-based classification head for incremental sessions. Experiments on multiple datasets show that this approach consistently outperforms existing FSCIL baselines and achieves state-of-the-art performance.

By Kyle Stein, Guillermo Francia, III Eman El-Sheikh, Andrew Arash Mahyari
arXiv AI
Aug 28

Beyond F1: Evaluating Coverage and Failure Recovery in AI Model Security Scanners

The paper evaluates three AI model security scanners—ModelScan, ModelAudit, and Fickling—using a benchmark of 170 Pickle and PyTorch artifacts from 145 families, 135 of which have binary security labels. It distinguishes coverage metrics such as non‑N/A coverage, analysis completion, and definitive security decisions, finding that ModelAudit achieved 100% definitive decisions, Fickling 81.5%, and ModelScan 49.6%. When a definitive judgment was made, ModelScan reached perfect precision, recall, and F1, while Fickling added no unique true positives beyond those found by the other tools.

By Qianlong Lan, Vinothini Pandurangan, Anuj Kaul, Indranil Sanyal
arXiv Machine Learning
Sep 24

Enhancing Multiclass Malware Classification in Resource-Constrained Environments

The paper presents a lightweight machine‑learning approach for multi‑class malware detection on resource‑constrained devices. Using a LightGBM classifier with SMOTE oversampling, SOM‑US undersampling, and Genetic‑Algorithm feature selection, the authors achieve 89.1 % accuracy on four malware families and 76 % on 16 individual malware types. A second Random‑Forest model further improves family classification to 91.2 % and individual classification to 78.7 %.

By Abdul Khalek Alve, Alif Rahman, Saadman Zaman, Sazzad Hossen Himel, Muhammad Iqbal Hossain