The paper investigates how six naturalistic and synthetic input perturbations affect decoder‑only language models at three levels: output behavior, hidden‑state geometry, and attention‑head function. Using GPT‑2 and Qwen2.5 checkpoints, the authors analyze layerwise geometry with centered kernel alignment and intrinsic dimension, and examine attention‑head responses in GPT‑2. They find that perturbation types produce distinct metric profiles that are not fully captured by output measures and vary across checkpoints, highlighting the need for multi‑level evaluation of robustness.
By Dun Li Chan, Emily Liu, Niyathi Allu, Christian Hoang
arXiv:2606. 08810v1 Announce Type: cross Abstract: Gaussian-corrupted sentence embeddings have no direct linguistic interpretation, yet continuous diffusion language models can generate fluent text from them.
By Zhicheng Du, Lan Ma
The paper investigates how six naturalistic and synthetic input perturbations affect decoder‑only language models at three levels: output behavior, hidden‑state geometry, and attention‑head function. Using GPT‑2 and Qwen2.5 checkpoints, the authors analyze layerwise geometry with centered kernel alignment and intrinsic dimension, and examine attention‑head responses in GPT‑2. They find that perturbation types produce distinct metric profiles that are not fully captured by output measures and vary across checkpoints, highlighting the need for multi‑level evaluation of robustness.
arXiv:2608.29748v1 Announce Type: new
Abstract: Speculative decoding accelerates autoregressive language model inference by having a lightweight draft model propose multiple candidate tokens, which a...
By Luxi Lin, Zhanpeng Zeng, Shuang Peng, Songwei Liu, Rongrong Ji
SimLens is a training‑free decoder that improves early‑layer predictions in large language models by keeping only the start token and a candidate answer token and performing a lightweight continuation through the remaining layers. It outperforms direct linear readouts, yielding higher accuracy on tasks such as ARC, BoolQ, and HeadQA with LLaMA‑7B and Vicuna‑7B. The method is extended to Linear SimLens for confidence estimation and combined into SimExit, a hybrid early‑exit mechanism that achieves significant speedups while maintaining accuracy.
By Ming Ma, Bowen Zheng, Zhongqiao Lin, Tianming Yang
arXiv:2601. 22947v2 Announce Type: replace-cross Abstract: Masked diffusion language models (MDLMs) generate text by unmasking tokens in parallel and have recently emerged as alternatives to autoregressive language models.
By Mengyu Ye, Keito Kudo, Ryosuke Takahashi, Jun Suzuki
arXiv:2606. 02544v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) have recently emerged as a promising alternative to autoregressive (AR) LLMs, offering faster inference through parallel or blockwise decoding.
By Junxia Cui, Haotian Ye, Runchu Tian, Hongcan Guo, Jinya Jiang, Haoru Li, Chaojie Ren, Yiming Huang, Kaijie Zhu, Zhongkai Yu, Kun Zhou, Jingbo Shang
arXiv:2609.37391v1 Announce Type: new
Abstract: Diffusion language models (DLMs) enable parallel generation by predicting and committing multiple tokens at each denoising step, yet they can generate...
By Kodai Kawamura, Kenji Kawaguchi, Anji Liu
arXiv:2607. 03502v1 Announce Type: cross Abstract: Frontier LLMs can perform multi-step reasoning over content-free filler tokens like dots or counting sequences, producing correct answers with no visible chain-of-thought (CoT).
By Kaley Brauer, Claudio Mayrink Verdun, Samuel Marks
arXiv:2607. 15655v1 Announce Type: cross Abstract: Masked diffusion language models (DLMs) enable parallel text generation by iteratively refining masked tokens, offering a promising alternative to autoregressive decoding.
By Yingqian Cui, Wei Deng, Lantao Mei, Hang Li, Charu C. Aggarwal, Hui Liu, Yue Xing
arXiv:2606. 12232v1 Announce Type: new Abstract: Masked diffusion language models (dLLMs) have recently emerged as a competitive alternative to autoregressive language models, with the promise of faster inference via parallel token generation.
By Stipe Frkovic, Metod Jazbec, Dan Zhang, Christian A. Naesseth, Ilija Bogunovic, Eric Nalisnick
arXiv:2607. 29378v1 Announce Type: cross Abstract: Large language models (LLMs) generate text by auto-regressively sampling the next token.
By Pirzada Suhail, Nagasai Saketh Naidu, Atanu R Sinha, Amit Sethi