arXiv Computation and Language By Shi-Qi Yan, Kai-Xuan Ding, Chao-Hong Tan, Qian Chen, Wen Wang, Xiangang Li, Zhen-Hua Ling

Short Horizons and Sparse Concepts: a Mathematical View of the Readout in the J-lens

Read the original on arXiv Computation and Language →

The paper presents a mathematical analysis of the Jacobian lens (J‑lens), a method for extracting verbalizable representations from language models. It treats the J‑lens as a first‑order causal transfer operator, showing that its Jacobian matrix serves as an optimal local linear approximation of downstream mappings and that its energy distribution is highly sparse, concentrating in diagonal pathways and critical positions. This sparse, short‑horizon structure explains why the J‑lens can effectively visualize concepts during a model’s reasoning process, and the authors propose a decoupling strategy that further improves its ability to read out correct intermediate concepts.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI
Jun 15

Jacobian Scopes: token-level causal attributions in LLMs

arXiv:2601. 16407v3 Announce Type: replace-cross Abstract: Large language models (LLMs) make next-token predictions based on clues present in their context, such as semantic descriptions and in-context examples.

By Toni J. B. Liu, Baran Zadeo\u{g}lu, Nicolas Boull\'e, Rapha\"el Sarfati, Gurbir Arora, Christopher J. Earls
arXiv AI
Jul 20

Verbalizable Representations Form a Global Workspace in Language Models

arXiv:2607. 15495v1 Announce Type: cross Abstract: Out of everything the human brain processes, only a small fraction is consciously accessible, in the sense of being available for verbal report, deliberate control, and flexible reasoning.

By Wes Gurnee, Nicholas Sofroniew, Adam Pearce, Mateusz Piotrowski, Isaac Kauvar, Runjin Chen, Anna Soligo, Paul Bogdan, Euan Ong, Rowan Wang, Ben Thompson, David Abrahams, Subhash Kantamneni, Emmanuel Ameisen, Joshua Batson, Jack Lindsey
arXiv Machine Learning
Sep 16

Attention Mean Fields Predict Average Representation Dynamics and Reveal Context-Specific Computation

The paper presents a mean‑field analysis of attention in language models, defining an average attention kernel that propagates representations layer by layer. When conditioned on a whole corpus, the kernel predicts the average evolution of representation geometry; when conditioned on a single context, it predicts the expected geometry for that context. The difference between actual attention and the mean‑field prediction—called the mean‑field deviation—captures context‑specific computation, revealing how models diverge from average behavior during training and in few‑shot tasks.

By Micah Adler, John W. Byers, Mark Crovella
Hugging Face Trending Papers
Aug 10

Interpreting Language Model Hidden States at Scale

Lens methods interpret large language models (LLMs) by mapping intermediate activations to the output vocabulary, revealing how next-token predictions develop through the network. Trained lenses remain expensive: affine-translator parameters grow quadratically with model width, while exact, full-vocabulary Kullback--Leibler (KL) training dominates memory.