← Back to all news
arXiv AI

Language models recognize dropout and Gaussian noise applied to their activations

Language models can detect, localize, and even verbalize differences between dropout and Gaussian noise applied to their activations. In experiments on eight models ranging from 8 B to 36 B, many achieve near‑perfect accuracy in identifying which sentence was perturbed and which type of perturbation was used. The study shows that this ability emerges during pre‑training, correlates with perturbation strength, and can be learned in context, while incorrect labels degrade performance.

Related stories

arXiv Machine Learning
Sep 11

Detectable Only Where It Is Confounded: What Verified Duplication Counts Say About Membership Evidence in Language Models

The paper investigates whether language models can identify sentences from their training data by using exact duplication counts from publicly released corpora for two model families, OLMo‑2 and Pythia. It finds that for typical duplication levels, models show only a weak trace of exposure, with a rank correlation near –0.08, and that strong signals only appear when a sentence appears roughly a thousand times, at which point fame rather than memory dominates. The study also demonstrates that common membership tests can be misleading, as changing a single word does not alter the model’s preference, and that controlling for register can significantly improve detector performance.

By Arman Nik Khah
llms
More like this →
arXiv AI
Sep 4

The Geometry of Ignorance: LLMs Know When to Temper Bayesian Priors

The paper identifies a single direction in the unembedding matrix of large language models that encodes the unigram distribution of the training corpus, acting as a Bayesian prior when the model is uncertain. By projecting the final prediction state onto this direction, the authors derive a per‑token prior loading factor, λ, which decreases as context becomes more informative and decomposes predictions into a tempered prior and a context‑driven likelihood. Experiments across four model families (Llama, Qwen, Gemma, Pythia) show that larger models rely less on the prior in high‑context settings and that manipulating λ can steer predictions toward or away from the unigram prior in KL divergence.

By Toni J. B. Liu, Jiajun Bao, Yizhou Liu, Gurbir Arora, Nicolas Boull\'e, Rapha\"el Sarfati, Christopher J. Earls
llms
More like this →
arXiv Machine Learning
Aug 20

Compress and Forget: bitsandbytes Quantization Amplifies Proactive Interference in LLMs

The study investigates how post‑training quantization (PTQ) affects proactive interference (PI) in large language models. Using bitsandbytes, the authors compare FP16, INT8, and INT4/NF4 precision across three instruction‑tuned models and find that INT4 quantization markedly degrades accuracy under high interference, with INT8 also incurring a smaller penalty in two of the three models. The degradation is linked to increased same‑key intrusion errors and originates in the quantized transformer backbone rather than the output layer.

By Shayan Shahrabi-Farahani (Shahid Beheshti University, Tehran, Iran), Dara Rahmati (Shahid Beheshti University, Tehran, Iran)
llmsnlpefficiencybenchmarks
More like this →
arXiv AI
Oct 2

Verbalized and Internal Probabilities Are Coupled in Large Language Models

The paper investigates the relationship between a large language model’s internal probability distribution and its verbalized confidence statements. By systematically manipulating training and in‑context data, the authors show that both internal and verbalized probabilities are influenced by distributional and asserted uncertainty in the data. They find that verbalized probabilities align with internal ones beyond what would be expected if they tracked the same sources independently, indicating that verbalized confidence can serve as a probe of the model’s internal distribution.

By Sinead Williamson, Jiaxuan Li, Nick Foti, Russ Webb, Masha Fedzechkina
llms
More like this →
arXiv AI
Oct 16

Learning to Read Out: Unembedding Dynamics in Language Model Pretraining

The paper investigates when a language model actually learns task information versus when its readout (unembedding matrix) learns to express that information. Using a new parameter‑trajectory cross‑coding method, the authors find that hidden states can encode task data early in training, but the readout reorganizes only in a narrow window (0.4–1.1% of Pythia’s pretraining). By varying the readout learning rate, they show that faster readout learning reduces the gap between hidden‑state encoding and output expression without affecting final performance.

By Matteo He, William F. Shen, Alex Iacob, Andrej Jovanovic, Xinchi Qiu, Nicholas D. Lane
llms
More like this →