The paper investigates how different training strategies affect the prompt sensitivity of large language models. It reproduces and compares methods such as refined data construction and robustness objectives, finding that while robustness fine‑tuning improves over standard fine‑tuning and in‑context learning, the prompt gap remains large (40–57%). Notably, newer techniques like CoIN and PPCL often underperform a simple data‑construction approach that uses one template per batch, and diagnostics suggest that mixed‑template batches force the optimizer to reconcile conflicting updates rather than learn a prompt‑agnostic representation.
By Frederic Sadrieh, Michal \v{S}tef\'anik
The paper investigates how small lexical changes in prompts can cause large performance swings in large language models. Using a dataset of 132,000 prompt variants, the authors uncover a scaling law linking higher average task performance to lower variance and greater robustness. They identify domain-specific terminology and explicit action directives as key linguistic factors that stabilize prompts, and propose an automated Prompt-Refining Agent that reduces performance variance by 40.7% in code generation while maintaining or improving mean performance.
By Qipeng Xie, Zi Liang, Jiafei Wu, Yufei Chen, Weizheng Wang, Wenao Ma, Zhong Ming, Haiqin Yang, Kaishun Wu
The paper presents practical training recipes for looped language models, showing that a compute‑efficient pipeline can reduce the training budget from 7.7 T tokens to 310 B tokens while maintaining strong reasoning performance. In controlled experiments, a 1.4 B LoopLM outperforms a parameter‑matched dense model on 12 benchmarks, achieving significant gains on GSM8K, MATH, and DROP, and approaches a 3.9 B dense model at matched inference compute. The authors also provide a minimal recipe to convert pretrained dense models into looped ones, demonstrating improvements on Qwen3‑1.7B‑Base across multiple data regimes.
By Andrei Marchenko, Viacheslav Bezrukov, Oleg Kashurin, Inessa Fedorova, Dmitry Bocharov, Yuliana Shakhvalieva, Maria Tikhonova, Valerii Ternovskii
arXiv:2602.21189v3 Announce Type: replace-cross
Abstract: Pass@$k$ is a widely used performance metric for verifiable large language model tasks, including mathematical reasoning, code generation, an...
By Anas Barakat, Souradip Chakraborty, Khushbu Pahwa, Amrit Singh Bedi
The paper introduces a fine-grained method called interactions to analyze prompt sensitivity in large language models (LLMs). By decomposing output scores into nonlinear interactions, the authors show that subtle prompt changes can destabilize these interactions even when overall outputs stay unchanged. They propose an Interaction-based Prompt Sensitivity (IPS) metric and use it to evaluate 50 open-source LLMs, finding that supervised fine‑tuning, larger model scales, dense architectures, and few‑shot learning all reduce prompt sensitivity, primarily by stabilizing low‑order interactions.
By Ruiyang Qin, Qingzhuo Wang, Tian Wang, Zhihua Wei, Wen Shen
arXiv:2606. 12117v1 Announce Type: cross Abstract: Benchmark scores often misrepresent a large language model's (LLM's) knowledge, because they rely, e.
By Selen Erkan, Bastian Boll, Kristian Kersting, Bj\"orn Deiseroth, Letitia Parcalabescu