The paper introduces a method that represents language models as log‑likelihood vectors over prompt‑response pairs, enabling the construction of model maps that compare conditional distributions. Squared Euclidean distances in this vector space approximate KL divergence, and experiments show that these maps reveal global structure related to model attributes and task performance. The approach also captures systematic shifts from prompt changes, supports additive compositionality for predicting downstream scores, and offers PMI vectors to mitigate unconditional distribution effects, thereby aiding analysis and prediction of input‑dependent behavior.
arXiv:2510. 01163v2 Announce Type: replace Abstract: The factors driving the performance of in-context learning (ICL) in large language models (LLMs) remain poorly understood despite ICL's surprising effectiveness, enabling models to adapt to new tasks from only a handful of examples.
arXiv:2606. 05486v1 Announce Type: cross Abstract: Prompt ambiguity is a common source of failure in large language models, but is difficult to localize because it is a latent property of the prompt, while existing attribution methods are designed to explain observable outputs such as logits or generated tokens.
arXiv:2607. 08399v1 Announce Type: cross Abstract: Large language models process prompts by propagating activations through dozens of layers before generating a response.
The study investigates how post‑training of large autoregressive language models (ARMs) into masked diffusion models (MDMs) affects their internal computation. Across two 7B ARM‑MDM families and four diagnostic tasks, the authors find that MDMs retain much of the ARM’s high‑attribution pathways on prefix‑dominant tasks, but reorganize computation toward earlier layers on globally constrained tasks. Component‑level probes reveal that ARMs depend on sharply specialized components, whereas MDMs show weaker specialization and more diffuse output‑space alignment.
arXiv:2608. 16627v1 Announce Type: cross Abstract: Natural language explanations (NLEs) are increasingly used as inputs, for example, as few-shot rationales that influence model behavior in in-context learning (ICL).
arXiv:2606. 00544v1 Announce Type: new Abstract: Modern language-model fine-tuning typically pairs each prompt with a single response, even though many prompts admit multiple valid completions.
The paper investigates how combining soft prompts via task arithmetic can reduce reliance on confounding variables in classification models. It introduces Hybrid Prompt Arithmetic (HyPA), which merges task prompts with linearized confounder prompts to counteract spurious correlations. Experiments across multiple benchmarks show that HyPA consistently improves the robustness‑performance trade‑off under distribution shift, and analysis of hidden representations suggests it mitigates confounding by diminishing the influence of confounder signals.
arXiv:2608.28382v1 Announce Type: new Abstract: Users often ask large language models (LLMs) to report how confident they are, but it is unclear whether such linguistic confidence tracks the model's...
The study investigates how different prompt components affect language model responses in psychometric tests. By crossing five distinct baseline personas with five variants of each prompt element—persona wording, task instruction, item wording, and option symbol—the authors measure response shifts using the 1‑Wasserstein distance. Their analysis of 13 small open‑weight language models on the Big Five Inventory and Short Dark Triad reveals that task instruction and option symbol changes often cause more variation than paraphrasing the persona or item, with prompt artifacts explaining over 50% of the variation for many items.
arXiv:2607. 29378v1 Announce Type: cross Abstract: Large language models (LLMs) generate text by auto-regressively sampling the next token.
Large language models (LLMs) trained on next-token prediction exhibit remarkable in-context learning (ICL) abilities, yet the representations that support ICL remain poorly understood. We consider suc...
Contextual entrainment, which is a newly discovered phenomenon in large language models (LLMs), refers to the tendency of a model to assign higher probabilities to tokens that appear in its context. In this work, we extend this phenomenon from the token level to the sentence level by examining the per-token mean log-probability of a sentence instead of the probabilities of individual tokens.