arXiv AI June 4, 2026 By Zacharie Bugaud

Unpredictable Safety: Domain-Dependent Compliance and the Transparency Gap in Open-Weight LLMs

arXiv:2606. 04035v1 Announce Type: cross Abstract: We present a systematic study of domain-dependent safety behavior in open-weight LLMs: 7 standardized experiments across 7 ethical domains, testing 5 models (12B--70B) in 4,200 interactions with dual-judge validation.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI

Jul 23

Guardrails as Scapegoats: Auditing Unfaithful Safety Refusals in Tool-Augmented LLM Agents

arXiv:2607. 19449v1 Announce Type: cross Abstract: Evaluation frameworks for tool-augmented LLM agents focus overwhelmingly on capability metrics or explicit tool crashes, leaving silent infrastructure failures and HTTP 200 responses with empty, null, or malformed payloads largely unaudited.

By Aarushi Singh

llms agents benchmarks safety

arXiv AI

Jun 6

Safety Paradox: How Enhanced Safety Awareness Leaves LLMs Vulnerable to Posterior Attack

arXiv:2606. 05614v1 Announce Type: new Abstract: Large language models (LLMs) are rigorously aligned to refuse harmful requests, a process that inherently cultivates a latent capacity to evaluate and recognize unsafe content.

By Long P. Hoang, Hai V. Le, Shaoyang Xu, Wei Lu, Wenxuan Zhang

llms reinforcement-learning safety

arXiv AI

Jul 7

Not All Refusals Are Equal: How Safety Alignment Fails Cybersecurity at Scale

arXiv:2607. 02714v1 Announce Type: cross Abstract: There is no doubt that safety alignment is an essential step in LLM training.

By Vadym Hadetskyi, Dario Pasquini, Artem Sorokin

llms safety

arXiv AI

Aug 5

Single Canonical Prompts Underestimate LLM Safety's Surface-Form Sensitivity

arXiv:2608. 02665v1 Announce Type: cross Abstract: A benchmark score is a measurement instrument, yet most benchmarks read each item at a single canonical surface form.

By Yongxi Zhou, Junwei Yao, Yuanzhe Liu, Zihan Dong, Wenbo Ye, Jiaxi Wen, Lai Yun Choi

llms benchmarks safety

arXiv AI

Jul 9

Institutional Red-Teaming: Deployment Rules, Not Just Models, Causally Shape Multi-Agent AI Safety

arXiv:2607. 07695v1 Announce Type: new Abstract: We introduce institutional red-teaming, an evaluation methodology for testing deployment rules in multi-agent AI: hold the agents, objectives, and task state fixed, vary only one rule, and attribute the resulting change in collective behavior to that rule.

By Yujiao Chen

llms agents benchmarks safety

arXiv AI

Jun 2

When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystems

arXiv:2606. 00448v1 Announce Type: cross Abstract: LLM agents increasingly rely on community-contributed skills that expand an agent's operational capability set.

By Su Wang, Pin Qian, Yihang Chen, Junxian You, Xiaoyuan Wang, Xiaochong Jiang, Lifei Liu, Haoran Yu, Jingzhou Xu

llms agents benchmarks safety

Related stories

Guardrails as Scapegoats: Auditing Unfaithful Safety Refusals in Tool-Augmented LLM Agents

Safety Paradox: How Enhanced Safety Awareness Leaves LLMs Vulnerable to Posterior Attack

Not All Refusals Are Equal: How Safety Alignment Fails Cybersecurity at Scale

Single Canonical Prompts Underestimate LLM Safety's Surface-Form Sensitivity

Institutional Red-Teaming: Deployment Rules, Not Just Models, Causally Shape Multi-Agent AI Safety

When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystems