arXiv Machine Learning
Jul 3
HaloGuard 1.0: An Open Weights Constitutional Classifier for Multilingual AI Safety
arXiv:2607. 02079v1 Announce Type: cross Abstract: We present HaloGuard 1.
HaloGuard 2.0: An Open-Weights Constitutional Output Guard for Multilingual Response Safety
HaloGuard 2.0 is an open‑weight, streaming output guard that assigns a safe/unsafe probability to each response token in a single forward pass, allowing deployments to halt a stream once harm is detected. It builds on HaloGuard 1.0’s input‑guard constitution, training on a 478,532‑response corpus and a second corrective phase with 106,043 counterfactual responses to address a shortcut that previously caused the model to judge prompts instead of responses. The resulting 0.8 B‑parameter model achieves an 83.6 macro F1 on six English safety benchmarks—up 10.8 points over the prior version—while using 5–10× fewer parameters than the strongest full‑response guards, and its multilingual variant reaches 83.1 F1 across 17 languages.
Related stories
arXiv AI
Jul 20
Latent Fusion Jailbreak: Blending Harmful and Harmless Representations to Elicit Unsafe LLM Outputs
arXiv:2508. 10029v3 Announce Type: replace-cross Abstract: Safety-aligned large language models can still be manipulated through white-box interventions that modify their internal representations.
arXiv Machine Learning
Jul 30
Recover, Decode, Reguard: Guard-Agnostic Defense Amplification againstEncoded VLM Jailbreaks
arXiv:2607. 26574v1 Announce Type: cross Abstract: Safety classifiers ("guards") are the dominant black-box defense for vision-language models, yet they judge an input's surface form, not its meaning: a harmful request re-encoded as set theory, formal logic, a rare language, code, or an image of text slips past a guard that would block it in plain language -- the decode gap.
arXiv AI
Sep 1
Reachability-Based Capability Confinement for LLM Agents under Indirect Prompt Injection
arXiv:2608.30041v1 Announce Type: cross Abstract: Large language model agents place outputs from external skills into their execution context, allowing attacker-controlled data to influence later pri...
arXiv AI
Jul 24
When Are Reasoning-Based Guardrails Not Efficient? ResponseGuard: A Fast Vision-Language Guard for Real-Time Moderation
arXiv:2607. 21401v1 Announce Type: cross Abstract: A vision-language AI assistant returns its answer as a stream of generated tokens.
arXiv Machine Learning
Jul 14
Safe responses matter: Output-aware safety guardrail mitigate over-refusal in MLLMs
arXiv:2607. 09697v1 Announce Type: new Abstract: Existing safety mechanisms for multimodal large language models (MLLMs) face a fundamental trade-off between safety and utility.
arXiv AI
Oct 16
Language Is a Security Surface for Agent Guards
The study investigates whether the language of a user’s instruction affects the safety verdicts of agent guards. By translating 1,220 labeled steps from an English benchmark into German, French, Turkish, Swahili, and Amharic while keeping all other data identical, the authors find that many open, small, and purpose‑built guards alter their verdicts when the instruction is translated, with larger effects for rarer languages. The impact varies with guard size, with smaller models showing greater loss of recall and larger models (e.g., Gemini 3.8 Flash) remaining largely unchanged.
arXiv AI
Oct 28
ADTGuard: Adaptive Depth-Time Safeguard for Streaming Inference in Large Language Models
ADTGuard is a new safeguard for large language models that monitors each token before it is streamed to users. It uses Depth-Time Safety Dynamics to combine safety signals from multiple layers and a Causal Depth Router to select the most relevant layers for each prefix. With only response‑level labels, it refines token‑level training targets and achieves strong streaming detection performance with minimal parameters and latency.
arXiv AI
Sep 15
Overflip: Repetition-Induced Label Flips in Guardrail Models
Guardrail models, which screen malicious prompts in LLM services, often use lightweight Transformers with short context windows and bucketed positional encodings. The study identifies a new failure mode called Overflip, where repeating a prompt causes the guardrail’s prediction to flip from malicious to benign as the sequence length increases. Experiments on nine popular guardrails show that 5 models exhibit MAL→BEN flips on 100 prompts, with flip rates ranging from 8% to 92% and first flips occurring between 2.6k and 9.4k tokens, highlighting a gradual attention dispersion distinct from traditional attention‑dilution attacks.
arXiv AI
Oct 23
AdaGuard: An Adaptive Guard Model with User-defined Policies
AdaGuard is an adaptive guard model that evaluates language‑model agent trajectories against user‑defined policies. The authors created the AdaptiveSafety dataset, containing 10,939 training and 1,000 test examples with policy rules, counterfactuals, explanations, and violated rules. Using supervised pre‑training followed by the SafePO reinforcement‑learning algorithm, they built 0.6B, 4B, and 8B guard models, with the 4B version achieving 89.30% accuracy on AdaptiveSafety and 71.82% on DynaBench.
arXiv AI
Jul 23
ChannelGuard: Safe Models Do Not Compose into Safe Multi-Agent Systems
arXiv:2607. 19430v1 Announce Type: cross Abstract: Multi-agent LLM applications chain a planner, worker agents, a verifier, and a synthesizer, and every hop between agents is an unmonitored channel through which an adversary can smuggle instructions.
arXiv AI
Aug 11
Attack Ensembles Expose a Safety-Utility Trade-off in Black-Box Guard Defenses Against Encoded VLM Jailbreaks
arXiv:2607. 26574v2 Announce Type: replace-cross Abstract: Safety classifiers ("guards") are the dominant black-box defense for vision-language models, yet a guard judges an input's surface form, not its meaning: a harmful request re-encoded as set theory, formal logic, a classical language, code, or text rendered inside an image slips past a guard that would block it in plain language - the decode gap.