arXiv AI
Jun 11
Estimating Tail Risks in Language Model Output Distributions
arXiv:2604. 22167v2 Announce Type: replace-cross Abstract: Language models are increasingly capable and are being rapidly deployed on a population-level scale.
ProbGuard: Calibrated Safety Risk Estimation from LLM Output Distributions
arXiv:2608. 10621v1 Announce Type: new Abstract: Recent research on Large Language Model (LLM) safety has widely adopted guardrails to identify unsafe LLM outputs.
Related stories
arXiv AI
Jun 4
Inference-Time Vulnerability Beyond Shallow Safety: Alignment Along Generation Trajectories
arXiv:2606. 04778v1 Announce Type: new Abstract: Safety-aligned Large Language Models (LLMs) remain vulnerable to interventions during inference that redirect generation toward harmful outputs.
arXiv AI
Jun 19
SafeSpec: Fast and Safe LLM via Dynamic Reflective Sampling
arXiv:2606. 19755v1 Announce Type: cross Abstract: Speculative inference accelerates large language model (LLM) decoding but provides no inherent safety guarantees.
arXiv AI
4d ago
Controlled Decoding Attacks on Black-Box LLMs
arXiv:2609.36956v1 Announce Type: cross Abstract: Manipulating next-token probabilities during generation can bypass the safety alignment of large language models. Existing approaches, however, rely...
arXiv AI
Aug 26
NeuronGuard: Robust LLM Safety Alignment via Ablation-Aware Safety Signal Redistribution
NeuronGuard is a fine‑tuning defense for large language models that hardens them against both jailbreak and neuron‑level attacks. It redistributes safety signals across many neurons by identifying safety‑critical ones with per‑layer linear classifiers, enforcing refusal behavior when those neurons are ablated, and applying KL‑divergence regularization for consistency. A randomized gradient projection preserves task performance, and the authors provide a formal guarantee that NeuronGuard lowers the attack success rate upper bound, with experiments showing near‑zero success rates across multiple models and attack strategies.
arXiv AI
Jun 6
Safety Paradox: How Enhanced Safety Awareness Leaves LLMs Vulnerable to Posterior Attack
arXiv:2606. 05614v1 Announce Type: new Abstract: Large language models (LLMs) are rigorously aligned to refuse harmful requests, a process that inherently cultivates a latent capacity to evaluate and recognize unsafe content.
arXiv AI
Jul 20
Latent Fusion Jailbreak: Blending Harmful and Harmless Representations to Elicit Unsafe LLM Outputs
arXiv:2508. 10029v3 Announce Type: replace-cross Abstract: Safety-aligned large language models can still be manipulated through white-box interventions that modify their internal representations.
arXiv AI
Sep 1
SingProbe Technical Report
arXiv:2608.30703v1 Announce Type: cross Abstract: Runtime guardrails are essential for reliable large language model (LLM) deployment, yet existing approaches typically rely on independent, external...
Hugging Face Trending Papers
Jun 4
Safety Paradox: How Enhanced Safety Awareness Leaves LLMs Vulnerable to Posterior Attack
Large language models (LLMs) are rigorously aligned to refuse harmful requests, a process that inherently cultivates a latent capacity to evaluate and recognize unsafe content. In this work, we reveal that this advanced safety awareness inadvertently introduces a fatal vulnerability.
arXiv Machine Learning
Jun 25
RAS: Measuring LLM Safety Through Refusal Alignment
arXiv:2606. 25750v1 Announce Type: cross Abstract: Safety evaluation of large language models (LLMs) is commonly performed by querying models with unsafe or jailbreak prompts and judging whether their outputs violate a safety policy.
arXiv AI
Jun 10
Stop Early, Spend Less: Hidden-State Probes as a Practical Recipe for Streaming Moderation of LLM Outputs
arXiv:2606. 10487v1 Announce Type: cross Abstract: Deploying large language models in user-facing systems requires efficient output safety filtering.
arXiv Computation and Language
Sep 15
ForeSight: Enhancing Risk Monitoring via Early Safety Signal Distillation
arXiv:2609.13737v1 Announce Type: new Abstract: As large language models (LLMs) are increasingly deployed, the generation of harmful content has become a critical safety concern. Existing safeguards...