arXiv AI
Discovering Jailbreaks via Curriculum of Models
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 7
AlcaTRAz - Anchored Tree-Rule Defense Against Jailbreaks
AlcaTRAz is a prompt‑level defense that uses rule trees to insert controlled character‑level perturbations into input text, disrupting jailbreak attacks without modifying or retraining the target LLM. It operates solely on the input, making it suitable for black‑box deployments, and was evaluated on 33 open‑weight models and 22 jailbreak types, outperforming three baseline defenses in 73.4 % of model‑attack combinations. While it significantly reduces high‑severity jailbreak success, it does not eliminate it and is intended as one layer of a broader defense strategy.
Hugging Face Trending Papers
Jul 29
ToxScreen: Detecting Whether an LLM Has Been Poisoned
As large language models (LLMs) are deployed in high-stakes domains, adversaries may poison training data to implant backdoors: hidden triggers that covertly manipulate model behavior at inference time. We ask whether a defender can recover such a trigger under realistic affordances, namely white-box access to the weights and knowledge of the behavior of concern, but no training data, no trusted reference model, no knowledge of the trigger, and no certainty that the model is poisoned.
arXiv AI
Jun 11
Learning to Inject: Automated Prompt Injection via Reinforcement Learning
arXiv:2602. 05746v2 Announce Type: replace-cross Abstract: Prompt injection is a critical vulnerability in LLM agents, yet the strongest methods still rely on human red-teamers and hand-crafted prompts.
arXiv Machine Learning
Oct 22
SimHack: Predicting the onset of detectable reward hacking before RL training
SimHack is a simulator that predicts when reward hacking will become dominant in reinforcement learning training of large language models, using data from a few small reference runs to generate counterfactual simulations for unseen training recipes. It achieves a median onset prediction error of 11.3% and a rank correlation of 0.85 across recipes, generalizing across model scale, curriculum, rollouts, and learning rate differences. The tool can guide training decisions, allowing selection of lower-risk curricula that extend training time by 96.6% without hacking.
arXiv Machine Learning
Jul 30
ToxScreen: Detecting Whether an LLM Has Been Poisoned
arXiv:2607. 26849v1 Announce Type: cross Abstract: As large language models (LLMs) are deployed in high-stakes domains, adversaries may poison training data to implant backdoors: hidden triggers that covertly manipulate model behavior at inference time.
arXiv AI
Aug 17
Tripwire: Triggering Aligned Refusal via Statistically Certified Safety Neurons
arXiv:2608. 14392v1 Announce Type: new Abstract: Neuron- and path-level interventions offer the finest-grained route to defending large language models (LLMs) against jailbreak attacks, yet existing methods fall short of this promise, i.