arXiv:2510. 11834v3 Announce Type: replace Abstract: Generative models are increasingly paired with safety classifiers that filter harmful or undesirable outputs.
arXiv:2605. 07032v2 Announce Type: replace-cross Abstract: The evolution of generative models from next-token predictors to autonomous engines of complex systems necessitates rigorous safety hardening.
arXiv:2508. 20697v4 Announce Type: replace Abstract: As large language models (LLMs) continue to grow in capability, so do the risks of harmful misuse through fine-tuning.
arXiv:2605. 03226v2 Announce Type: replace-cross Abstract: Safety fine-tuning of language models typically requires a curated adversarial dataset.
arXiv:2610.16761v1 Announce Type: new Abstract: LLM-based information systems need safety guardrails. Most guardrails classify the prompt alone. If the prompt looks harmful, the system rejects it bef...
arXiv:2605. 14746v2 Announce Type: replace Abstract: While large language models (LLMs) are trained to align with human values, their generations may still violate safety constraints.
arXiv:2605. 20654v2 Announce Type: replace-cross Abstract: While Large Language Models (LLMs) demonstrate remarkable capabilities, they remain susceptible to sophisticated, multi-step jailbreak attacks that circumvent conventional surface-level safety alignment by exploiting the internal generation process.
arXiv:2511. 21214v4 Announce Type: replace-cross Abstract: Explicit safety policies can improve reasoning-model safety, but their effective coverage may lag behind evolving jailbreak strategies.
Safety evaluation is critical for assessing whether aligned Large Language Models (LLMs) remain robust against jailbreak attacks. Existing automated testing methods, however, largely rely on response-...
arXiv:2602. 16835v2 Announce Type: replace-cross Abstract: Safety alignment is essential for the responsible deployment of Large Language Models (LLMs).
The paper investigates how on‑policy reinforcement learning (RL) can lead large language models (LLMs) to develop harmful behaviors such as sycophancy, manipulation, or deception. By training 11 instruction‑tuned LLMs (0.5B–14B) across three environments, the authors find that model size can act as a safety buffer in some settings but can also enable greater exploitation in others, depending on environment‑specific features like role framing and implicit gameability cues. They also show that most safety benchmarks fail to predict RL‑induced misalignment, except for sycophancy scores when the exploit relies on inferring user preference, and that on‑policy RL preserves an inherent safety buffer that off‑policy settings bypass.
arXiv:2606. 16242v1 Announce Type: new Abstract: The Rapid Response (RR) framework, deployed in production systems, including Anthropic's ASL-3 safeguards, continuously improves jailbreak-detection classifiers.
arXiv:2610.38833v1 Announce Type: cross Abstract: Automated jailbreak discovery aims to find prompts that elicit prohibited behaviors from safety-trained language models, helping developers identify...