arXiv AI July 8, 2026 By Konstantin Berlin

Full-range Binary Classifier Calibration for Stable Model Updates in Production

arXiv:2607. 05481v1 Announce Type: cross Abstract: Detection models running in adversarial environments face a malicious distribution that drifts rapidly while the benign distribution stays comparatively stable, so teams retrain and redeploy constantly to stay ahead of new threats.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

safety

arXiv AI

Jun 19

Calibration Without Comprehension: Diagnosing the Limits of Fine-Tuning LLMs for Vulnerability Detection in Systems Software

arXiv:2606. 20502v1 Announce Type: cross Abstract: Whether LLMs scoring well on vulnerability benchmarks genuinely reason about security or merely pattern-match on contaminated data remains unresolved.

By Arastoo Zibaeirad, Marco Vieira

llms fine-tuning benchmarks

arXiv AI

Aug 17

Regime-Conditional Verification: Correctness Estimation for Adapting and Monitoring Safety Classifiers

arXiv:2608. 14089v1 Announce Type: new Abstract: Safety classifiers deployed with large language models often fail for two reasons: their decisions reflect the policy learned during training rather than the deployer's desired policy, and their performance degrades as deployment traffic evolves.

By Thiago Sandoval, Ufuk Topcu

llms fine-tuning benchmarks safety

arXiv AI

Aug 28

Beyond F1: Evaluating Coverage and Failure Recovery in AI Model Security Scanners

The paper evaluates three AI model security scanners—ModelScan, ModelAudit, and Fickling—using a benchmark of 170 Pickle and PyTorch artifacts from 145 families, 135 of which have binary security labels. It distinguishes coverage metrics such as non‑N/A coverage, analysis completion, and definitive security decisions, finding that ModelAudit achieved 100% definitive decisions, Fickling 81.5%, and ModelScan 49.6%. When a definitive judgment was made, ModelScan reached perfect precision, recall, and F1, while Fickling added no unique true positives beyond those found by the other tools.

By Qianlong Lan, Vinothini Pandurangan, Anuj Kaul, Indranil Sanyal

benchmarks

arXiv Machine Learning

Aug 12

ProbGuard: Calibrated Safety Risk Estimation from LLM Output Distributions

arXiv:2608. 10621v1 Announce Type: new Abstract: Recent research on Large Language Model (LLM) safety has widely adopted guardrails to identify unsafe LLM outputs.

By Xinzhe Huang, Biwu Yao, Kedong Xiu, Mengnan Zhao, Di Wang, Puning Zhao, Tianhang Zheng

llms safety

arXiv AI

Jun 3

High-Precision APT Malware Attribution with Out-of-Scope Resilience

arXiv:2606. 03523v1 Announce Type: cross Abstract: Early attribution of Advanced Persistent Threat (APT) activity can help defenders prioritise investigation, select countermeasures, and reduce the impact of an intrusion.

By Peter Williams, Adam Sobey, Erisa Karafili

arXiv Machine Learning

Jul 21

When Drift Detectors cry Wolf: False Alarm Rates in continuous ML Monitoring

arXiv:2607. 17336v1 Announce Type: new Abstract: Drift detection is a core component of production machine learning monitoring systems, where detectors are used to compare incoming data with a reference distribution and trigger alerts when changes occur.

By Raj Shekhar Singh

safety

One email a morning, machine-written

Related stories

Calibration Without Comprehension: Diagnosing the Limits of Fine-Tuning LLMs for Vulnerability Detection in Systems Software

Regime-Conditional Verification: Correctness Estimation for Adapting and Monitoring Safety Classifiers

Beyond F1: Evaluating Coverage and Failure Recovery in AI Model Security Scanners

ProbGuard: Calibrated Safety Risk Estimation from LLM Output Distributions

High-Precision APT Malware Attribution with Out-of-Scope Resilience

When Drift Detectors cry Wolf: False Alarm Rates in continuous ML Monitoring