The paper investigates how diffusion language models maintain factual fidelity when generating text from tables, comparing masked diffusion models (MDLM, LLaDA) to autoregressive models (GPT‑2, GPT‑Neo) on the ToTTo dataset. Using a retrieval‑based few‑shot protocol, the study finds that while diffusion models score higher on reference similarity (BLEU), they exhibit lower source grounding and different failure modes—autoregressive models drift from the table, whereas diffusion models stay on topic but swap entities. The authors conclude that reference‑based metrics alone are insufficient for assessing faithfulness in table‑to‑text generation.
By Zaynab Merimi, Noussayba El Marrakchi, Arthur Satouf
The paper critiques generative perplexity (gen‑PPL) as an unreliable metric for evaluating unconditional text generation, showing that it can be minimized by trivial, non‑diverse outputs. The authors demonstrate this by constructing simple samplers that achieve low gen‑PPL yet produce incoherent text, and they propose alternative evaluation tools—including distributional discrepancies, diversity diagnostics, and a compression‑based KL divergence measure—to better assess generative language models. They argue that future progress in diffusion language models should rely on a suite of metrics that capture diverse aspects of generated text quality.
By Antonio Franca, Alexander Tong
arXiv:2606. 06474v1 Announce Type: cross Abstract: Discrete diffusion language models generate text by iteratively denoising an entire response in parallel.
By Paul J\"unger, Justin Lovelace, Linxi Zhao, Dongyoung Go, Kilian Q. Weinberger
The study compares continuous diffusion language models (CDLMs) and speculative drafters by training them from scratch on unconditional OpenWebText generation at lengths 16, 128, and 1024. Using LangFlow for CDLMs and adapted DSpark Markov and recurrent-head architectures for standalone generation, the authors evaluate generation perplexity (GenPPL) and MAUVE similarity. Results show LangFlow outperforms both DSpark variants at longer lengths (128 and 1024), while DSpark-RNN leads at length 16; recurrent memory and teacher probability matching improve metrics but the longer‑sequence gap remains, highlighting iterative full‑sequence refinement as a key advantage.
By Yujian Chen, Hao Jiang, Guang Cheng, Weijian Luo, Tianyang Hu
arXiv:2605. 17932v2 Announce Type: replace-cross Abstract: Prompt compression reduces inference cost and context length in large language models, but prior evaluations focus mainly on autoregressive architectures.
By Sterling Huang, Abigayle Brown, Jiyoo Noh, Jiakang Xu, Wantong Huo, Kaung Myat Kyaw, Jonathan Chan
arXiv:2606. 08417v1 Announce Type: cross Abstract: Diffusion and continuous flow-based language models have emerged as the leading non-autoregressive alternatives to language modeling.
By Antonio Franca, Alexander Tong
The paper introduces Seq-Drifting, a new framework that enables diffusion language models to generate an entire text sequence in a single forward pass. By applying a drift field to continuous token representations, the model learns a direct mapping from Gaussian noise to text, using attraction to guide outputs toward contextually plausible tokens and repulsion to prevent repetition and collapse. Experiments show that Seq-Drifting achieves a perplexity of 36.01 on LM1B and performs competitively in unconditional, conditional, translation, summarization, and reasoning tasks.
By Jiayu Liu, Ruizhi Zhao, Tianyun Ji, Yee Whye Teh, Zhenya Huang, Ning Miao
The paper introduces probe guidance, a technique that leverages frozen internal states of a diffusion model to generate a guidance signal without requiring an extra forward pass during inference. This method improves continuous diffusion language models, achieving state‑of‑the‑art results on unconditional generation and enhancing performance on multiple‑choice question answering for a 1.7B model. The authors also use probes to analyze autoguidance, revealing that the weak model must originate from a low‑entropy training region to align dynamics with the strong model.
By Rohit Dilip, Tianrong Chen, Yuyang Wang, David Van Valen, Joshua Susskind, Miguel Angel Bautista
arXiv:2610.26951v1 Announce Type: new
Abstract: Retrieval-augmented generation (RAG) is the dominant paradigm for grounding large language models in external corpora, but its similarity-driven, chunk...
By Debopriyo Banerjee, Owais Aijaz, Ahmer Tabassum, Zhuohan Xie, Preslav Nakov, Veselin Stoyanov
arXiv:2606. 11552v1 Announce Type: cross Abstract: Large language models (LLMs) achieve remarkable performance across a wide range of tasks, but their autoregressive decoding process incurs substantial inference costs due to inherently sequential token generation.
By Lexington Whalen, Yuki Ito, Ryo Sakamoto
arXiv:2603.13911v2 Announce Type: replace
Abstract: A language model can give the wrong answer even when the correct answer is decodable from its intermediate states. To study this gap between decoda...
By Valeria Ruscio, Keiran Thompson
arXiv:2610.17266v1 Announce Type: new
Abstract: Diffusion language models are a promising and more efficient alternative to autoregressive models. However, most RL post-training techniques are not ea...
By Vaibhav Singh, Eugene Belilovsky