arXiv Machine Learning

PVF:Understanding AI Vulnerability Against SDCs

arXiv:2405. 01741v4 Announce Type: replace-cross Abstract: Reliability of AI systems is a fundamental concern for the successful deployment and widespread adoption of AI technologies.

arXiv AI

Aug 12

On Understanding, Identifying, and Mitigating Vulnerabilities in Agentic Large Language Models

arXiv:2608. 10530v1 Announce Type: cross Abstract: Large Language Models (LLMs) have undergone a shift from stateless conversational interfaces to autonomous agents capable of multi-step planning, tool invocation, code execution, and maintaining persistent memory.

By Md Jafrin Hossain, Mohammad Arif Hossain, Nirwan Ansari

llms agents safety

arXiv AI

Jun 19

Calibration Without Comprehension: Diagnosing the Limits of Fine-Tuning LLMs for Vulnerability Detection in Systems Software

arXiv:2606. 20502v1 Announce Type: cross Abstract: Whether LLMs scoring well on vulnerability benchmarks genuinely reason about security or merely pattern-match on contaminated data remains unresolved.

By Arastoo Zibaeirad, Marco Vieira

llms fine-tuning benchmarks

arXiv Machine Learning

Jul 3

From Lab to Reality: A Practical Evaluation of Deep Learning Models and LLMs for Vulnerability Detection

arXiv:2512. 10485v2 Announce Type: replace-cross Abstract: Vulnerability detection methods based on deep learning (DL) have shown strong performance on benchmark datasets, yet their real-world effectiveness remains underexplored.

By Chaomeng Lu, Bert Lagaisse

llms benchmarks

arXiv AI

Aug 11

AIVV: Neuro-Symbolic LLM Agent-Integrated Verification and Validation for Trustworthy Autonomous Systems

arXiv:2604. 02478v2 Announce Type: replace Abstract: Deep learning models excel at detecting anomaly patterns in normal data.

By Jiyong Kwon, Ujin Jeon, Sooji Lee, Guang Lin

llms agents

arXiv AI

Jun 26

Hierarchical Fault Detection and Diagnosis for Transformer Architectures

arXiv:2604. 28118v2 Announce Type: replace-cross Abstract: Transformers now underpin critical AI systems across industry and research.

By Sigma Jahan, Saurabh Singh Rajput, Tushar Sharma, Mohammad Masudur Rahman

llms benchmarks

arXiv Machine Learning

Aug 5

Evaluation Blindness: How Silent Measurement Failures Corrupt AI Systems from Training to Deployment

arXiv:2608. 02786v1 Announce Type: new Abstract: AI systems can fail silently.

By Priyanka Bajaj (Independent Researcher)

llms fine-tuning benchmarks

arXiv Machine Learning

Aug 11

Learning to Triage Vulnerability Reports from Program Analysis: An Empirical Study in Node.js

arXiv:2510. 20739v2 Announce Type: replace-cross Abstract: Program analysis tools often produce large volumes of candidate vulnerability reports that require costly manual review, creating a practical challenge: how can security analysts prioritize the reports most likely to be true vulnerabilities?

By Ronghao Ni, Aidan Z. H. Yang, Min-Chien Hsu, Nuno Sabino, Limin Jia, Ruben Martins, Darion Cassel, Kevin Cheang

llms benchmarks

arXiv AI

Jun 8

Beyond the Black Box: Interpretability of Agentic AI Tool Use

arXiv:2605. 06890v3 Announce Type: replace Abstract: AI agents are promising for high-stakes enterprise workflows, but dependable deployment remains limited because tool-use failures are difficult to diagnose and control.

By Hariom Tatsat, Ariye Shater

llms agents benchmarks safety

arXiv Machine Learning

Jun 5

LLMs for Secure Hardware Design and Related Problems: Opportunities and Challenges

arXiv:2605. 10807v4 Announce Type: replace-cross Abstract: The integration of Large Language Models (LLMs) into Electronic Design Automation (EDA) and hardware security is rapidly reshaping the semiconductor industry.

By Johann Knechtel, Ozgur Sinanoglu, Ramesh Karri

llms agents benchmarks safety

arXiv AI

Jul 1

A Lifecycle and Application-Stack Survey of Large Language Model Vulnerabilities: Attacks, Risks, Defenses, and Open Problems

arXiv:2606. 31639v1 Announce Type: cross Abstract: Large language models are no longer only text generators.

By Seyed Bagher Hashemi Natanzi, Bo Tang

llms agents robotics safety

arXiv AI

Jul 14

DeepBias: Adaptive In-depth Probing of Social Biases in LVLMs

arXiv:2607. 11228v1 Announce Type: cross Abstract: While Large Vision-Language Models (LVLMs) demonstrate remarkable capabilities, they remain highly susceptible to embedded social biases.

By Anqi Li, Jie Zhang, Zhongqi Wang, Songkai Xue, Jiahao Wang, Shiguang Shan, Xilin Chen

llms agents reinforcement-learning multimodal benchmarks safety

arXiv Machine Learning

Jul 1

An Empirical Study of Security Calibration in Large Language Models for Code

arXiv:2606. 31159v1 Announce Type: cross Abstract: Large Language Models (LLMs) are rapidly transforming software development, yet their use in security-critical contexts raises a key question: do models know when their generated code is insecure?

By Mohammed Latif Siddiq, Md. Nafiu Rahman, Joanna C. S. Santos

llms benchmarks

Related stories