The paper investigates how six naturalistic and synthetic input perturbations affect decoder‑only language models at three levels: output behavior, hidden‑state geometry, and attention‑head function. Using GPT‑2 and Qwen2.5 checkpoints, the authors analyze layerwise geometry with centered kernel alignment and intrinsic dimension, and examine attention‑head responses in GPT‑2. They find that perturbation types produce distinct metric profiles that are not fully captured by output measures and vary across checkpoints, highlighting the need for multi‑level evaluation of robustness.
By Dun Li Chan, Emily Liu, Niyathi Allu, Christian Hoang
arXiv:2606. 08810v1 Announce Type: cross Abstract: Gaussian-corrupted sentence embeddings have no direct linguistic interpretation, yet continuous diffusion language models can generate fluent text from them.
By Zhicheng Du, Lan Ma
The paper investigates how six naturalistic and synthetic input perturbations affect decoder‑only language models at three levels: output behavior, hidden‑state geometry, and attention‑head function. Using GPT‑2 and Qwen2.5 checkpoints, the authors analyze layerwise geometry with centered kernel alignment and intrinsic dimension, and examine attention‑head responses in GPT‑2. They find that perturbation types produce distinct metric profiles that are not fully captured by output measures and vary across checkpoints, highlighting the need for multi‑level evaluation of robustness.
arXiv:2608.29748v1 Announce Type: new
Abstract: Speculative decoding accelerates autoregressive language model inference by having a lightweight draft model propose multiple candidate tokens, which a...
By Luxi Lin, Zhanpeng Zeng, Shuang Peng, Songwei Liu, Rongrong Ji
SimLens is a training‑free decoder that improves early‑layer predictions in large language models by keeping only the start token and a candidate answer token and performing a lightweight continuation through the remaining layers. It outperforms direct linear readouts, yielding higher accuracy on tasks such as ARC, BoolQ, and HeadQA with LLaMA‑7B and Vicuna‑7B. The method is extended to Linear SimLens for confidence estimation and combined into SimExit, a hybrid early‑exit mechanism that achieves significant speedups while maintaining accuracy.
By Ming Ma, Bowen Zheng, Zhongqiao Lin, Tianming Yang
arXiv:2601. 22947v2 Announce Type: replace-cross Abstract: Masked diffusion language models (MDLMs) generate text by unmasking tokens in parallel and have recently emerged as alternatives to autoregressive language models.
By Mengyu Ye, Keito Kudo, Ryosuke Takahashi, Jun Suzuki