arXiv Computation and Language
Predicting Linear Relational Decodability Without the Relation
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
Oct 7
World Properties without World Models: Distributional Associations and the Interpretation of Decoding Results from Language Models
arXiv:2603.04317v2 Announce Type: replace-cross Abstract: A growing literature shows that variables can be linearly decoded from the activations of large language models (LLMs). These range from prop...
arXiv AI
Jun 3
Relational Linearity is a Predictor of Hallucinations
arXiv:2601. 11429v2 Announce Type: replace-cross Abstract: Hallucination is a central failure mode of language models (LMs).
arXiv AI
Oct 16
Knowledge That Is Linear Before Fine-Tuning Generalizes After It
The study investigates how well language models can generalize from fine‑tuned facts by distinguishing active facts—those that support further learning—from inert facts. Using a novel relabeling experiment (e.g., mapping Paris → Europe to Paris → crab), the authors fine‑tune models on a subset of subjects and test whether the learned mapping extends to unseen subjects sharing the same original answer. Across eleven models and 70 relations, they find a strong correlation (Pearson r = 0.87–0.92) between the linear readability of the original relation in pretrained representations and the model’s ability to generalize, a relationship that persists after controlling for various factors and remains robust to label randomization and semantic category changes.
Hugging Face Trending Papers
Oct 6
U-Space: Uncovering When and Why Uncertainty Arises in Language Models
Large language models are informing decisions with ever-higher stakes. As the consequences of their errors grow, a central question becomes harder to ignore: how much can we trust an individual answer...
arXiv AI
Jul 20
Verbalizable Representations Form a Global Workspace in Language Models
arXiv:2607. 15495v1 Announce Type: cross Abstract: Out of everything the human brain processes, only a small fraction is consciously accessible, in the sense of being available for verbal report, deliberate control, and flexible reasoning.
Hugging Face Trending Papers
Oct 15
Knowledge That Is Linear Before Fine-Tuning Generalizes After It
The paper distinguishes between active facts, which help a language model generalize beyond mere recall, and inert facts, which do not. By relabeling answers to a known relation (e.g., Paris → Europe becomes Paris → crab) and fine‑tuning on a subset of subjects, the authors test whether the model can transfer the new label to unseen subjects sharing the same original answer. Across eleven models and 70 relations, the linear readability of the original relation in pretrained representations strongly predicts this generalization, with a Pearson correlation of 0.87–0.92, even after controlling for various factors.