The study investigates how lexical perturbations—such as keyboard noise, character swaps, and filler insertion—affect large language models (LLMs) on reasoning benchmarks. Four open-weight instruction-tuned models and frontier models were evaluated, revealing that character-level perturbations significantly reduce accuracy, especially on multi-step reasoning tasks, while filler insertion has minimal impact. The authors attribute this asymmetry to Attention Diversion, where fragmented subword tokenization draws disproportionate attention in middle and final transformer layers; they demonstrate that both token content and attention allocation are coupled, making it difficult for inference-time repair strategies to fully recover performance.
By Jiaqian Zhu, Yang Zhang, Junhua Ding, Xiaowei Yu
arXiv:2605.30913v2 Announce Type: replace-cross
Abstract: Large language models (LLMs) are increasingly deployed in conversational settings where user phrasing ranges from polite to casual to toxic,...
By Soorya Ram Shimgekar, Agam Goyal, Amruta Parulekar, Joshua Chen, Yian Wang, Navin Kumar, Hari Sundaram, Eshwar Chandrasekharan, Koustuv Saha
arXiv:2603.13911v2 Announce Type: replace
Abstract: A language model can give the wrong answer even when the correct answer is decodable from its intermediate states. To study this gap between decoda...
By Valeria Ruscio, Keiran Thompson
The paper introduces Variance‑Calibrated Modulation (VCM), a training‑free pre‑decoding technique that reshapes language model probability distributions before truncation. VCM uses two dynamic mechanisms: a Contextual Searchlight via PMI to suppress stopwords and highlight context‑relevant tokens, and an Adaptive Self‑Debiasing that applies scale‑invariant penalization based on real‑time logit standard deviation. Experiments on open‑ended generation, factual QA, and mathematical reasoning show that VCM consistently reduces the likelihood trap, improving diversity, coherence, and reasoning accuracy with minimal computational cost.
By Yuanhao Ding, Meimingwei Li, Esteban Garces Arias, Matthias A{\ss}enmacher, Christian Heumann, Chongsheng Zhang
arXiv:2603. 18482v2 Announce Type: replace-cross Abstract: Standard decoding strategies for text generation, including top-$k$, nucleus sampling, and contrastive search, select tokens based on likelihood, restricting outputs to high-probability regions.
By Esteban Garces Arias, Nurzhan Sapargali, Christian Heumann, Matthias A{\ss}enmacher
The study investigates how textual neural models degrade when inputs contain noise such as typos, OCR errors, or dropped words. It finds that model performance decline is largely consistent across architectures under word‑level noise but diverges under character‑level noise, a difference attributed to tokenization rather than architecture. By applying a short contrastive training recipe, diverse encoders converge to a common robustness curve, enabling prediction of a model’s noise resilience and the ability to enhance robustness at specific noise scales through targeted training.
By Yefan Tao, Gerald Friedland, Luyang Kong
The paper presents a reference‑free framework for detecting hallucinations in Sinhala‑to‑English neural machine translation. It introduces a 45,000‑sample synthetic dataset created via five linguistically motivated corruption strategies and a semantic rescue mechanism that uses character‑level similarity. Using a fine‑tuned mDeBERTa‑v3 model, the authors achieve a token‑level F1 of 0.841 and demonstrate that the detector relies on the Sinhala source, not on surface artefacts, with an AUROC of 0.970 on a source‑disjoint test set. The study also benchmarks eight NMT systems, showing that hallucination rates vary by an order of magnitude across architectures.
By Navam Obeysekara, Nevidu Jayatilleke
The paper introduces a black‑box, inference‑time diagnostic for low‑resource Automatic Post‑Editing (APE) that distinguishes whether poor performance is due to insufficient training data or inconsistent training signals. By varying an edit‑distance penalty and analyzing the resulting TER‑vs‑λ curve and confidence‑based constraint ordering, the authors identify two failure modes—Binary Collapse and Confident Miscalibration—across multiple language pairs. The diagnostic also suggests practical next steps, such as applying a static constraint for immediate accuracy gains, and the authors release new English‑Sinhala and English‑Tamil APE datasets with accompanying code.
By Isuru Wijesiri, Nisansa de Silva, Kavindu Warnakulasuriya, Aloka Fernando, Surangika Ranathunga
The paper investigates how safety‑aligned large language models (LLMs) can still be tricked by jailbreaks or over‑refuse benign requests. By analyzing residual‑stream updates in decoder‑only Transformers, the authors find that harmfulness is more readily decoded from attention updates—especially in early layers—while the MLP component is primarily responsible for writing refusal responses. Post‑training enhances MLP‑driven refusal writing without improving harmfulness decodability, and causal interventions confirm that downstream MLP updates are crucial for the refusal shift caused by upstream attention changes. Building on these insights, the authors propose Sentinel, a system that combines an attention‑derived perception score with an MLP‑derived decision score to better detect jailbreaks and over‑refusals across multiple models.
By Zixuan Weng, Haoqi He, Weifei Jin, Yue Huang, Yang Bai
The paper investigates how six naturalistic and synthetic input perturbations affect decoder‑only language models at three levels: output behavior, hidden‑state geometry, and attention‑head function. Using GPT‑2 and Qwen2.5 checkpoints, the authors analyze layerwise geometry with centered kernel alignment and intrinsic dimension, and examine attention‑head responses in GPT‑2. They find that perturbation types produce distinct metric profiles that are not fully captured by output measures and vary across checkpoints, highlighting the need for multi‑level evaluation of robustness.
By Dun Li Chan, Emily Liu, Niyathi Allu, Christian Hoang
The paper investigates how six naturalistic and synthetic input perturbations affect decoder‑only language models at three levels: output behavior, hidden‑state geometry, and attention‑head function. Using GPT‑2 and Qwen2.5 checkpoints, the authors analyze layerwise geometry with centered kernel alignment and intrinsic dimension, and examine attention‑head responses in GPT‑2. They find that perturbation types produce distinct metric profiles that are not fully captured by output measures and vary across checkpoints, highlighting the need for multi‑level evaluation of robustness.
The paper introduces the Adversarial Surface-Form Robustness Dataset (ASRD), which contains 2,100 prompts across seven surface‑form families to test large language models on non‑canonical inputs. Five open‑weight models were evaluated on 10,500 responses, and a Quad‑State Evaluation Rubric categorized each response as harmful compliance, safe response, comprehension failure, or indeterminate. Results show that emoji and invisible Unicode variations produce few comprehension failures, while leetspeak, encoded wrappers, and hybrid transformations lead to high comprehension failure rates and varied harmful compliance percentages.
By Pavan Maddula