arXiv AI July 3, 2026 By Mona Schirmer, Metod Jazbec, Alexander Timans, Christian Naesseth, Maja Waldron, Eric Nalisnick

Online Safety Monitoring for LLMs

Read the original on arXiv AI →

arXiv:2607. 02510v1 Announce Type: new Abstract: Despite alignment training, LLMs remain prone to generating unsafe outputs at deployment time.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI

Jun 19

LLM agent safety, multi-turn red-teaming, jailbreak benchmarks, adversarial robustness, safety-critical systems

arXiv:2606. 20408v1 Announce Type: cross Abstract: Large language model (LLM) agents are increasingly proposed as supervisory components for safety-critical systems, yet their robustness under sustained, adaptive adversarial pressure remains poorly characterized.

By Hanwool Lee, Dasol Choi, Bokyeong Kim, Seung Geun Kim, Haon Park

llms agents benchmarks safety

arXiv AI

Jul 7

NRT-Bench: Benchmarking Multi-Turn Red-Teaming of LLM Operator Agents in Safety-Critical Control Rooms

arXiv:2606. 20408v3 Announce Type: replace-cross Abstract: Large language model (LLM) agents are increasingly proposed as supervisory components for safety-critical systems, yet their robustness under sustained, adaptive adversarial pressure remains poorly characterized.

By Hanwool Lee, Dasol Choi, Bokyeong Kim, Haon Park, Seung Geun Kim

llms agents benchmarks safety

arXiv AI

Jun 10

Stop Early, Spend Less: Hidden-State Probes as a Practical Recipe for Streaming Moderation of LLM Outputs

arXiv:2606. 10487v1 Announce Type: cross Abstract: Deploying large language models in user-facing systems requires efficient output safety filtering.

By Huizhen Shu, Xuying Li, Piao Xue

llms safety

arXiv AI

Aug 12

REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems

arXiv:2608. 10669v1 Announce Type: new Abstract: Large language model (LLM) agents combine language-based reasoning with external tools to perform complex tasks.

By Zixing Chen, Xingyuan Liu, Jie Zhu, Huaixia Dou, Shuo Jiang, Junhui Li, Lifan Guo, Feng Chen, Chi Zhang

llms agents multimodal benchmarks safety

arXiv AI

Jul 24

Code Monitor Red Teaming for Public-Test-Passing Code

arXiv:2607. 20852v1 Announce Type: new Abstract: Visible tests are a common gate for LLM-generated code, but passing them does not certify specification correctness.

By Junchi Liao, Jiawen Deng, Fuji Ren

llms safety

arXiv Machine Learning

Aug 4

Why Formal Monitors Fail: Attack Distribution Entropy as a Coverage Bound for LTL-Based LLM Agent Safety

arXiv:2608. 01388v1 Announce Type: cross Abstract: Runtime safety monitors based on Linear Temporal Logic (LTL) and finite automata (FSA) are increasingly deployed to intercept unsafe tool-call sequences in LLM agents.

By Ruiyang Zhang

llms agents safety

Related stories

LLM agent safety, multi-turn red-teaming, jailbreak benchmarks, adversarial robustness, safety-critical systems

NRT-Bench: Benchmarking Multi-Turn Red-Teaming of LLM Operator Agents in Safety-Critical Control Rooms

Stop Early, Spend Less: Hidden-State Probes as a Practical Recipe for Streaming Moderation of LLM Outputs

REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems

Code Monitor Red Teaming for Public-Test-Passing Code

Why Formal Monitors Fail: Attack Distribution Entropy as a Coverage Bound for LTL-Based LLM Agent Safety