arXiv AI August 11, 2026 By Alizishaan Khatri, Dun Li Chan

Do All LLMs Know When They're Being Harmful? A Reproducibility Study of Latent-Space Safety Probes Across Model Families

arXiv:2608. 08029v1 Announce Type: cross Abstract: Khatri et al.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI

Jun 19

Calibration Without Comprehension: Diagnosing the Limits of Fine-Tuning LLMs for Vulnerability Detection in Systems Software

arXiv:2606. 20502v1 Announce Type: cross Abstract: Whether LLMs scoring well on vulnerability benchmarks genuinely reason about security or merely pattern-match on contaminated data remains unresolved.

By Arastoo Zibaeirad, Marco Vieira

llms fine-tuning benchmarks

arXiv Machine Learning

Jul 7

ProbeLogits: Kernel-Level LLM Inference Primitives for AI-Native Operating Systems

arXiv:2604. 11943v3 Announce Type: replace-cross Abstract: An OS kernel that runs LLM inference internally can read the model's own next-token logit distribution before any text is generated, and act on it as a governance primitive.

By Daeyeon Son

llms agents fine-tuning benchmarks safety

arXiv AI

Aug 5

Single Canonical Prompts Underestimate LLM Safety's Surface-Form Sensitivity

arXiv:2608. 02665v1 Announce Type: cross Abstract: A benchmark score is a measurement instrument, yet most benchmarks read each item at a single canonical surface form.

By Yongxi Zhou, Junwei Yao, Yuanzhe Liu, Zihan Dong, Wenbo Ye, Jiaxi Wen, Lai Yun Choi

llms benchmarks safety

arXiv AI

Jun 11

Layer-Isolated Evaluation: Gating the Deterministic Scaffold of a Production LLM Agent with a No-LLM, Regression-Locked Test Harness

arXiv:2606. 11686v1 Announce Type: cross Abstract: End-to-end task-success is the dominant way to evaluate LLM agents, but one aggregate number tells you that an agent regressed, not where.

By Sawyer Zhang, Alexander Wang, Sophie Lei

llms agents safety

arXiv AI

Aug 6

Diagnosing Tool-Selection Reasoning in LLM Agents with Canary Tools

arXiv:2608. 04719v1 Announce Type: new Abstract: Agent evaluations tell us that a model picked the wrong tool, but rarely why.

By Atul Anand, Sourav Chattaraj

llms agents benchmarks safety

arXiv AI

Aug 5

Your Agentic LLMs Secretly Encode Latent Signals of Indirect Prompt-Injection Exposure

arXiv:2608. 02657v1 Announce Type: cross Abstract: Agentic LLMs are vulnerable to indirect prompt injection (IPI) attacks, e.

By Jianshuo Dong, Yiming Liu, Maosen Zhang, Nan Deng, Xu Peng, Xiaoping Zhang, Tianwei Zhang, Jie Zhang, Han Qiu

llms agents

Related stories

Calibration Without Comprehension: Diagnosing the Limits of Fine-Tuning LLMs for Vulnerability Detection in Systems Software

ProbeLogits: Kernel-Level LLM Inference Primitives for AI-Native Operating Systems

Single Canonical Prompts Underestimate LLM Safety's Surface-Form Sensitivity

Layer-Isolated Evaluation: Gating the Deterministic Scaffold of a Production LLM Agent with a No-LLM, Regression-Locked Test Harness

Diagnosing Tool-Selection Reasoning in LLM Agents with Canary Tools

Your Agentic LLMs Secretly Encode Latent Signals of Indirect Prompt-Injection Exposure