The paper evaluates TypeSafe AI’s Jev safety judge against GPT and GLM judges and local classifiers using a controlled adversarial test harness. Jev shows strong robustness on rule‑based tasks like input moderation, with minimal bypasses, but performs less well on judgment‑based tasks such as phishing detection, where it misses a higher proportion of phishing emails and is more prone to false positives. The study highlights Jev’s speed advantage and varying resilience depending on task type.
arXiv:2610.16814v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as judges to assign scalar scores that drive downstream decision-making, including tasks such as p...
The paper introduces DEFER1, a deterministic-first enforcement system for large‑language‑model based multi‑agent systems that uses 28 checks to block most attacks and refers only a small fraction to human judges. In tests across four domains, DEFER1 reduces attack success from about 30% to roughly 3%, with 78% of attacks blocked deterministically and only a quarter reaching the judges. The study highlights that rules effectively handle clear policy violations while judges address ambiguous intent, but also reveals weaknesses such as a risk‑score gate that misclassifies many proposals.
The paper presents a study on securing large language model (LLM) inference services against adversarial use, such as jailbreaking, denial‑of‑service, and distillation attacks. It introduces a structural causal model to generate a realistic, labeled dataset of user sessions, including coordinated multi‑account campaigns and varying label observability. Using this dataset, the authors train a gradient‑boosted detector that achieves near‑perfect binary classification on oracle labels but shows lower performance on operational labels, and they demonstrate a simple thresholded engine that improves attack‑type attribution accuracy.
arXiv:2606. 25487v1 Announce Type: cross Abstract: Almost every paper on LLM jailbreaks and prompt injection reports an attack-success rate (ASR), and that number is assigned not by people but by an automated judge: either a safety classifier trained for the task, or a general chat model prompted to grade.
The paper introduces a prompt‑injection detection framework for email assistants that models attacks as a chain of stages. It combines a text detector, stage‑specific verifiers, rule‑based risk signals, user intent consistency checks, and a logistic decision policy. Experiments on five benchmarks show the framework outperforms pretrained detectors, achieving a mean F1 of 0.406 versus 0.216, and demonstrate that training on benign emails resembling attacks reduces false alarms.
Counter‑GEO‑Bench is a new benchmark that evaluates how well defenses can stop large language models from producing misinformation when faced with generative engine‑optimized (GEO) content. It contains 247 human‑verified queries paired with both information‑preserving and information‑distorting GEO rewrites, and measures attack success rate, false positives, and answer quality across three victim LLMs. The study shows that existing off‑the‑shelf defenses reduce attack success by at most 5.7 %, while a lightweight baseline called C‑GEO Guard cuts success by 47.6 % with minimal loss of utility.
JevAdvBench introduces the first adversarial benchmark for reinforcement‑learning‑based calibrated decision (RLCD) models, providing 812 typed questions across 66 scenarios and a black‑box attack suite of 9,744 single‑edit variants. The benchmark evaluates attacks by comparing each perturbed decision to the model’s own clean decision and to an identical re‑run, revealing that rewording changes decisions by only 1.2 percentage points while certain injected opinions can flip 12.1% of decisions and lower confidence below 0.8 in 38% of cases. These findings demonstrate that RLCD models can be significantly misled by seemingly innocuous input edits, underscoring the need to treat the state as untrusted in applications.
Counter‑GEO‑Bench is a new defense benchmark that tests how well systems can resist misinformation generated by generative engine optimization (GEO). It contains 247 human‑verified queries paired with both information‑preserving and information‑distorting GEO rewrites, and evaluates defenses on attack success rate, false positives, and answer quality across three large language models. Existing off‑the‑shelf defenses reduce attack success by at most 5.7%, while a lightweight baseline, C‑GEO Guard, cuts it by 47.6% with minimal loss of utility.
Almost every paper on LLM jailbreaks and prompt injection reports an attack-success rate (ASR), and that number is assigned not by people but by an automated judge: either a safety classifier trained for the task, or a general chat model prompted to grade. The judge is rarely checked.
The paper examines the vulnerability of open‑weight large language model safeguards that aim to block harmful usage by assuming new harmful behavior arises only through fine‑tuning. It demonstrates that simple, low‑cost attacks—abliteration and prefilling—can significantly increase attack success rates across multiple benchmarks, and proposes an abliteration‑resistant tuning (ART) method that mitigates these attacks. The study shows that safeguarding defenses must be evaluated against a broader range of attack strategies beyond fine‑tuning.
The paper introduces ADVERSA, an automated red‑teaming framework that evaluates large language model safety over multiple turns by tracking continuous compliance trajectories instead of binary jailbreak outcomes. Using a fine‑tuned 70B attacker model and a structured 5‑point rubric, the authors conduct controlled experiments on three frontier victim models, measuring guardrail degradation and judge reliability through a triple‑judge consensus. Results show a 26.7% jailbreak rate with most breaches occurring early, and the study documents inter‑judge agreement, attacker drift, and attacker refusals as key factors affecting safety assessment.
arXiv:2609.01487v1 Announce Type: cross Abstract: Skill-augmented agents load reusable skills as persistent runtime context, improving task performance but also giving malicious skills a durable chan...