arXiv AI

Beyond Token Positions: Safety Alignment Across Denoising Steps in Diffusion Language Models

This paper investigates safety alignment in diffusion large language models (dLLMs), which generate text via iterative denoising instead of left‑to‑right decoding. By tracking token distributions and commitment decisions across denoising steps, the authors find that refusal signals are concentrated early in the denoising process and at leading response positions, and that early committed tokens strongly influence the final safety outcome. They introduce Refusal‑Aware Early Commitment (RAEC), a training‑free decoding method that preserves early refusal signals, and demonstrate that RAEC reduces attack success rates on LLaDA and Dream while largely maintaining utility.

arXiv AI

6d ago

Why Jailbreaks Succeed in Diffusion Language Models: An Energy Landscape Analysis

The paper proposes a framework that explains why jailbreak attacks succeed against diffusion-based large language models (dLLMs) by viewing safety alignment as shaping a denoising energy landscape. It identifies two attack strategies—obscuring the query’s safety disposition at initialization or forcing the denoising path across an energy barrier mid‑trajectory—and introduces three training‑free detection signals that monitor initial safety disposition and kinetic energy in complementary subspaces. Experiments on several dense and sparse dLLMs show that these signals complement each other, and any attack that evades detection also fails to produce harmful content.

By Thong Bach, Dung Nguyen, Thao Minh Le, Truyen Tran

llms diffusion safety

arXiv AI

Jun 4

Supportive Token Revealing for Fast Diffusion Language Model Decoding

arXiv:2606. 04236v1 Announce Type: cross Abstract: Discrete diffusion language models can generate text efficiently by updating multiple masked positions in parallel, but this parallelism introduces a quality-latency trade-off.

By Giries Abu Ayoub, Mario Barbara, Llu\'is Pastor-P\'erez, Tanja Bien, Aneesh Barthakur, Alaa Maalouf, Loay Mualem

llms diffusion benchmarks

arXiv Machine Learning

Jul 13

Optimizing Against Safety Representations: Activation-Guided Adversarial Suffixes and the Geometry of Refusal

arXiv:2607. 08883v1 Announce Type: new Abstract: Behavioral alignment in large language models often masks fragile internal safety representations.

By Ege \c{C}akar, Hannah Guan, Kayden Kehe

llms safety

arXiv Machine Learning

Jun 4

When Autoregressive Consistency Hurts Safety Alignment

arXiv:2606. 04168v1 Announce Type: new Abstract: Safety alignment in large language models (LLMs) is fragile in part because it is often shallow: fine-tuning mainly reshapes the model's behavior near the first few output tokens.

By Bochen Lyu, Yiyang Jia, Xiaohao Cai, Zhanxing Zhu

llms fine-tuning safety

arXiv AI

Jul 1

The Geometry of Refusal: Linear Instability in Safety-Aligned LLMs

arXiv:2606. 22686v2 Announce Type: replace-cross Abstract: Modern Large Language Models (LLMs) rely on extensive safety alignment, yet the mechanistic basis of refusal remains opaque.

By Shivam Ratnakar, Kartikeya Vats

llms multimodal safety

arXiv Machine Learning

4d ago

Rethinking Soft Tokens for Parallel Decoding in Diffusion Language Models

arXiv:2609.37391v1 Announce Type: new Abstract: Diffusion language models (DLMs) enable parallel generation by predicting and committing multiple tokens at each denoising step, yet they can generate...

By Kodai Kawamura, Kenji Kawaguchi, Anji Liu

llms rag diffusion benchmarks

arXiv AI

Jun 4

MaskForge: Structure-Aware Adaptive Attacks for Jailbreaking Diffusion Large Language Models

arXiv:2606. 04027v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) generate text by iteratively denoising partially masked sequences under bidirectional context, exposing a safety surface distinct from autoregressive LLMs.

By Yingzi Ma, Zhengyue Zhao, Xiaogeng Liu, Minhui Xue, Yue Zhao, Chaowei Xiao

llms diffusion reinforcement-learning benchmarks safety

arXiv AI

Jun 8

Latent-space Attacks for Refusal Evasion in Language Models

arXiv:2605. 21706v2 Announce Type: replace Abstract: Safety-aligned language models are trained to refuse harmful requests, yet refusal behavior can be suppressed by steering their internal representations.

By Giorgio Piras, Raffaele Mura, Fabio Brau, Maura Pintor, Luca Oneto, Fabio Roli, Battista Biggio

llms multimodal benchmarks safety

arXiv AI

Jul 20

Latent Fusion Jailbreak: Blending Harmful and Harmless Representations to Elicit Unsafe LLM Outputs

arXiv:2508. 10029v3 Announce Type: replace-cross Abstract: Safety-aligned large language models can still be manipulated through white-box interventions that modify their internal representations.

By Wenpeng Xing, Bohan Yang, Mohan Li, Chunqiang Hu, Haitao Xu, Ningyu Zhang, Bo Lin, Meng Han

llms multimodal benchmarks safety

arXiv Machine Learning

Jun 3

Backdooring Masked Diffusion Language Models

arXiv:2605. 19262v2 Announce Type: replace Abstract: Masked diffusion language models (MDLMs) are emerging as a compelling new paradigm for text generation, but their training-time security remains largely unexplored.

By Daniel Yiming Cao, Chengzhong Wang, Sheng-Yen Chou, Chengyu Huang, Pin-Yu Chen, Shengwei An

llms diffusion fine-tuning

arXiv Computation and Language

Aug 28

Dependency-Aware Revocable Decoding for Efficient Diffusion Large Language Model Inference

The paper introduces Dependency-Aware Revocable Decoding (DARD), a training‑free framework for diffusion large language models that separates tokens into masked, candidate, and unmasked states. DARD verifies candidate tokens using a selective context that excludes less reliable tokens and adaptively regulates their influence on subsequent decoding. Experiments on 12 textual and multimodal benchmarks across three open‑source dLLMs show that DARD improves the speed‑quality Pareto frontier, achieving a 2.71× speedup and a 4.35‑point CIDEr gain over Saber on Flickr30K.

By Wooje Park, Insu Lee, Minyoung Noh, Jaeyun Jang, Sungmin Lee, Kyuhong Shim, Byonghyo Shim

llms diffusion multimodal benchmarks

arXiv AI

Aug 10

Diffusion LLMs as Targets and Adversaries: Mechanistic Safety Exploits

arXiv:2608. 07430v1 Announce Type: cross Abstract: Diffusion Large Language Models (DLLMs) replace autoregressive next-token prediction with iterative parallel denoising, yet their internal safety mechanisms remain poorly understood.

By Elena Dumitrescu, Gert Lek, Lydia Y. Chen, J\'er\'emie Decouchant

llms diffusion efficiency multimodal safety

Related stories