← Back to all news
arXiv AI By Alexander Herzog, Aliai Eusebi, Lorenzo Cavallaro

On the Reliability and Stability of Selective Methods in Malware Classification Tasks

Read the original on arXiv AI →

The paper examines whether modern drift‑adaptive malware classifiers provide reliable and stable confidence estimates when faced with distribution shifts. It introduces Aurora, a framework that evaluates confidence quality and operational resilience, and presents metrics that extend beyond point‑in‑time performance to assess stability over time. The study finds that state‑of‑the‑art models exhibit fragility across datasets with varying drift severity, indicating a need to revisit underlying assumptions.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Jul 28

EXE-Bench: Ranking the Tradeoffs of AI-based Windows Malware Detectors for Real-World Usability

arXiv:2607. 24177v1 Announce Type: cross Abstract: Due to the lack of systematic evaluations, we are not yet able to determine which AI-based Windows malware detector to deploy in production, since existing evaluations (i) differ in terms of data used for both training and testing; (ii) do not consider temporal analysis to showcase whether models withstand the passage of time; (iii) avoid security evaluations with adversarial attacks that could highlight their brittleness against content-injection attacks; and (iv) neglect the computational requirements for deployment, risking slow inference on endpoints.

By Andrea Ponte, Daniel Gibert, Matous Kozak, Dmitrijs Trizna, Maura Pintor, Battista Biggio, Fabio Roli, Luca Demetrio
benchmarkssafety
More like this →
Hugging Face Trending Papers
Jul 2

Beyond Gradient-Based Attacks: Adversarial Robustness and Explainability Stability in Cybersecurity Classifiers

Adversarial attacks on cybersecurity classifiers pose a dual threat: degrading predictions and destabilising the SHAP-based explanations that security analysts rely on to understand and triage alerts. We extend our prior MLP conference study to Random Forest and XGBoost across four tabular security datasets (phishing URLs, UNSW-NB15, NF-ToN-IoT, HIKARI-2021), evaluating five attacks including three black-box methods applicable to non-differentiable tree models.

benchmarkssafety
More like this →