arXiv:2608. 15448v1 Announce Type: cross Abstract: Large language models increasingly rely on sampling as a driver of their own improvement, making the fidelity of their learned distributions more critical than ever.
arXiv:2609.37594v1 Announce Type: new Abstract: Uncertainty estimates tell us how unsure a model is, but not why. Without knowing which parts of an input influences a model's uncertainty, we cannot t...
arXiv:2607. 00510v1 Announce Type: new Abstract: Knowing which training examples drive outputs is fundamental to auditing, correcting, and understanding language models, yet for modern LLMs this remains expensive, approximate, and largely post-hoc.
arXiv:2608.30315v1 Announce Type: new Abstract: Token embeddings are the basic representational units that connect discrete tokens with continuous computation in language models. Although modern lang...
The paper argues that large language models cannot achieve perfect reliability for any task, even with unlimited scale. It establishes that each generative task has an inherent reliability ceiling set by how much output uncertainty can be resolved from observable context, with a resolvable part that can be improved by more context and a subjective part tied to task ambiguity. The authors derive a scaling law showing that performance is limited by the scarcer resource—either training data or model capacity—and explain how this law explains phenomena such as retrieval augmentation and catastrophic forgetting.
arXiv:2603.18908v5 Announce Type: replace Abstract: Independently trained language models often learn compatible late-stage representations, despite differences in training objectives, architectures,...
arXiv:2506. 07406v3 Announce Type: replace-cross Abstract: Understanding the internal representations of large language models (LLMs) is a central challenge in interpretability research.
The study investigates how textual neural models degrade when inputs contain noise such as typos, OCR errors, or dropped words. It finds that model performance decline is largely consistent across architectures under word‑level noise but diverges under character‑level noise, a difference attributed to tokenization rather than architecture. By applying a short contrastive training recipe, diverse encoders converge to a common robustness curve, enabling prediction of a model’s noise resilience and the ability to enhance robustness at specific noise scales through targeted training.
arXiv:2410. 12341v4 Announce Type: replace-cross Abstract: As AI-generated content increasingly populates the web, generative AI models are at growing risk of being trained on their own outputs, a process known as AI autophagy.
arXiv:2605. 22864v2 Announce Type: replace Abstract: The maximum softmax probability (MSP) represents a default approach when evaluating uncertainty quantification for language model generation with structured output.
arXiv:2512.00763v2 Announce Type: replace-cross Abstract: Adaptive and non-Euclidean optimizers often outperform Euclidean methods such as stochastic gradient descent (SGD) in language modeling by a...
The paper introduces Perturbation, a method that treats representations in language models as learning conduits rather than activation patterns. By fine‑tuning a model on a single adversarial example and observing how this perturbation spreads to other inputs, the approach avoids geometric assumptions and does not identify representations in untrained models. In trained models, Perturbation uncovers structured transfer across multiple linguistic scales, indicating that language models generalize along representational lines and acquire linguistic abstractions through experience.
arXiv:2605. 13026v2 Announce Type: replace-cross Abstract: Masked diffusion models (MDMs) have emerged as a promising alternative to autoregressive models (ARMs) for language modeling.