← Back to all news
arXiv Computation and Language

Written Once, Read Differently How Language Models Own Answers Steer Their Next Ones

The paper investigates how a language model’s first answer to a yes/no question influences its subsequent answer to a different question about the same entity, a phenomenon termed self‑commitment. By fixing the initial answer and measuring changes in the second across 3,600 question pairs for 22 open‑weight models (0.5B–72B parameters), the authors find that each answer writes a commitment direction into the residual stream, and the similarity of these directions predicts answer coupling. They show that ablating a shared direction reduces coupling significantly, while the reading circuit for this direction varies with model size, instruction tuning, and prompts, and that sparing the attention sink during ablation preserves low perplexity cost in most models.

Related stories

arXiv Computation and Language
Sep 22

MechaTerp-TRACE: A Novel Approach for Component Ablation Analysis in Language Models

MechaTerp-TRACE is a new framework that systematically ablates individual components of language models to measure their causal contribution to producing a named entity. By applying TRACE to thirteen instruction‑tuned dense decoder models, the study finds that a small set of positionally fixed components consistently carry the most influence across models and prompts, while the remaining support is evenly distributed. This suggests that entity knowledge is largely embedded in generic generation machinery rather than in isolated, findable components.

By Brandon Colelough, Davis Bartels, Madeline Bittner, Dina Demner-Fushman
llmsnlpbenchmarkssafety
More like this →
arXiv AI
Oct 12

The One-Word Census: Answer-Choice Conformity Across 105 Language Models

The study examines how 105 language models choose a single word from a large set of valid options, finding that 46% of the time they all pick the same word, "serendipity." Using 96 prompts that ask for one-word answers across various categories, the researchers measure convergence and each model’s share of the consensus. They observe that post‑training and persona‑tuned models diverge more, while heavily post‑trained assistants are more conformist, and that the field’s answer choices are more concentrated than human norms in most categories.

By Tapan Parikh
llmsragfine-tuning
More like this →
arXiv Computation and Language
2d ago

A Capitulation Direction Emerges With Scale and Is Causally Load-Bearing Where Validation Is Strong: Eight Models, Two Families, and Five Measurement Hazards

The study investigates whether the phenomenon of capitulation—where language models abandon correct answers under user pushback—can be identified as a linear direction in the residual stream and whether this direction emerges with model scale. Analyzing eight instruction‑tuned models from 0.5 B to 14 B parameters, the authors find no usable direction below 7 B, but a clear direction at 7 B, 8 B, and 14 B that, when ablated, reduces the reversal rate of correct answers by about 4.8 percentage points. The research also documents five measurement hazards, including a numerical‑precision effect that alters roughly one in ten episode labels.

By Saad Aamir, Muhammad Awais Bin Adil
llms
More like this →
arXiv AI
Aug 25

Answer First, Reason Later: When Commitment Order Costs Accuracy in Diffusion Language Models

The paper studies how the order in which tokens are committed in masked diffusion language models affects accuracy. It finds that when the final answer is committed before the preceding reasoning (an answer‑first trajectory), accuracy can suffer compared to unrestricted decoding, especially on tasks like GSM8K and MATH‑500. Experiments with controlled token positions show that delaying the answer token can improve performance, indicating that commitment order influences the context and output allocation of the model.

By Jewon Yeom, Jaewon Sok, Seonghyeon Park, Jeongjae Park, Hwiyeong Lee, Taesup Kim
llmsdiffusion
More like this →