arXiv AI

The Shape of Wisdom: How Language Models Converge on an Answer

arXiv AI
Sep 12

From Parameters to Answers: How LLMs Retrieve and Use Their Internal Knowledge

The paper investigates how large language models (LLMs) such as Qwen, Llama, and Gemma use internal knowledge when answering questions. By performing layer‑wise interventions on the hidden state after the question, the authors compare how different request directions (pair‑conditioned vs. global) and answer types (noun, adjective, code) influence the model’s routing of information. The study finds that the influence of request direction varies across models and layers, with some models showing a sustained routing effect while others do not, highlighting distinct patterns of early readability, causal steering, and later content dependence.

By Wenkang Wei, Yuan Fang, Renhe Jiang, Hong Cheng, Xingtong Yu
arXiv AI
Sep 30

Locating Answer-Correctness Signals in Frozen Large Language Models

The paper investigates where and how large language models encode signals that indicate answer correctness. By examining hidden states, token probabilities, residual-stream features, attention, and their combinations, the authors find that correctness signals are concentrated in the answer span and that different signal families complement each other. Fusing these signals improves robustness, especially under distribution shifts, and can be used to control retrieval in downstream tasks.

By Yuansen Liu, Yixuan Tang, Anthony Kum Hoe Tung
arXiv Machine Learning
6d ago

MicroLens: Read, Trace, and Rewire the Residual Stream with Two Lenses

MicroLens introduces an interactive viewer that overlays two decoding lenses—logit and Jacobian—onto the residual stream of a language model, allowing users to read, trace, and edit token representations on a layers‑by‑tokens grid. Experiments on Gemma‑3‑4B‑it show that the logit lens directs token outputs while the Jacobian lens influences how later tokens interpret information, enabling targeted edits. The tool reveals that key information (e.g., state or country names) can be read and manipulated early in the stream, and that correct answers often appear in the residual stream before reasoning is generated, with the Jacobian lens detecting them slightly earlier than the logit lens.

By Nika Zahedi, Fangyi Zhu, Anna Wu
arXiv Computation and Language
Oct 16

Where Is the "Self" of a Large Language Model? -- From Self-Index to Self-Reference

The paper investigates whether a large language model (LLM) possesses a notion of self by distinguishing between two biological analogues: a self‑index that separates the model’s own material from external material, and self‑reference that allows the model to recognize its own content. Using Qwen3.8‑27B, the authors demonstrate that the model’s internal representations consistently differentiate its own messages across all layers, that the model claims ownership of its messages with a measurable correlation to the self‑index, and that manipulating internal states can reverse or diminish this self‑attribution. The study further shows that the model’s self‑reference ability is enhanced when it engages in a “thinking mode” and that the self‑index influences the model’s performance on arithmetic tasks.

By Yuchen Li, Xintong Li, Zongqi Fan, Ken-tye Yong, Nguyen H. Tran
arXiv Computation and Language
Oct 19

On the Tip of the Tongue: First-Token Cueing as a Behavioral Probe of Recall Failures in LLMs

The paper investigates why large language models often answer factual questions incorrectly in closed‑book settings. It shows that many errors are due to first‑token recall failures: the model commits to an incorrect first token but can complete the correct answer once the right token is provided. Using a First‑Token Cueing probe, the authors demonstrate that correcting just the first token recovers a significant portion of errors across 16 open models and improves accuracy on multilingual benchmarks.

By Jaewon Sok, Jewon Yeom, Seonghyeon Park, Jeongjae Park, Hwiyeong Lee, Taesup Kim
Hugging Face Trending Papers
Aug 19

Readable, Faithful, Used: Three Dissociable Properties of Demographic Identity in a Language Model

The study investigates how demographic identity is represented in a language model, using representational similarity analysis against Pew survey data across 169 demographic cells. It finds that standard last‑token read‑outs underestimate the model’s fidelity, while specific attention heads (notably L11 H16) capture demographic structure more accurately, though race‑based types remain weak. Causal interventions reveal that high fidelity does not guarantee causal use, and a 128‑dimensional probe of a single head improves alignment with survey truth but fails to recover per‑question group ordering.

arXiv AI
Oct 2

The Geometry of Contextual Relations: Language Models Address Facts by Order of Mention

The paper investigates how large language models (LLMs) encode and retrieve facts based on their order of mention in a context. By measuring changes in hidden states when shifting questions from one fact to another, the authors identify an "ordinal vector" that points to a fact’s position in the list rather than its name. Across several models (Qwen, Gemma, Llama), these ordinal addresses are ordered, steerable, low‑rank, and emerge in mid‑layers, suggesting that LLMs locate facts by their mention order, mirroring aspects of human recall.

By Yufa Zhou
arXiv Computation and Language
Nov 4

Written Once, Read Differently How Language Models Own Answers Steer Their Next Ones

The paper investigates how a language model’s first answer to a yes/no question influences its subsequent answer to a different question about the same entity, a phenomenon termed self‑commitment. By fixing the initial answer and measuring changes in the second across 3,600 question pairs for 22 open‑weight models (0.5B–72B parameters), the authors find that each answer writes a commitment direction into the residual stream, and the similarity of these directions predicts answer coupling. They show that ablating a shared direction reduces coupling significantly, while the reading circuit for this direction varies with model size, instruction tuning, and prompts, and that sparing the attention sink during ablation preserves low perplexity cost in most models.

By Marcus Armstrong