arXiv Computation and Language By Xiaoyi Yu, Enver Sangineto, Pei Fu, Fiorenzo Parascandolo, Wenhui Tan, Ruikang Zhang, Rita Cucchiara, Ruihua Song, Jian Luan

Informed Masking: Structure-Aware Perturbation for Reinforcement Learning in Diffusion Large Language Models

Read the original on arXiv Computation and Language →

Informed Masking (IM) is a new technique for aligning Diffusion Large Language Models (dLLMs) with Reinforcement Learning (RL). It identifies a systematic upstream/downstream token structure in dLLM rollouts and shows that masking downstream tokens creates better subproblems for likelihood estimation. When integrated into three state‑of‑the‑art dLLM RL methods on LLaDA‑8B‑Instruct, IM yields up to 2.01%, 8.68%, and 5.77% relative average gains on math and planning benchmarks while improving training stability.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Machine Learning
Nov 6

PBPO: Prefix-Block Policy Optimization for Diffusion Language Models

The paper introduces Prefix-Block Policy Optimization (PBPO), a method that adapts block-level policy optimization to Diffusion Large Language Models (dLLMs). PBPO constructs a prefix-block state and uses a single model evaluation to create a factorized one‑step surrogate for block likelihood, allowing surrogate importance weighting and clipping to be applied separately to each token. Experiments on LLaDA-family models across six benchmarks and three generation lengths show that PBPO achieves competitive performance compared to existing dLLM reinforcement learning methods.

By Katherine Tieu, Dongqi Fu, Si Zhang, Hanqing Zeng, Tianxin Wei, Chenglin Wei, Yinglong Xia, Hong Li, Jingrui He