arXiv AI June 9, 2026 By Javier Mar\'in

How Transformers Reject Wrong Answers: Rotational Dynamics of Factual Constraint Processing

arXiv:2603. 13259v2 Announce Type: replace-cross Abstract: When a decoder-only transformer is forced to process matched correct and incorrect single-token continuations of a factual query, the two pathways through hidden-state space diverge in a specific way: displacement vectors from the query-only representation maintain approximately equal magnitude but rotate apart in direction.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv Machine Learning

Aug 12

Off-Axis, On Purpose: Where a Transformer Computes Concepts and Why it Does So

arXiv:2608. 10251v1 Announce Type: cross Abstract: A transformer's answer lives on one axis: the direction its unembedding reads.

By Mark Oskin

llms benchmarks

arXiv AI

Jul 7

Input Pathways Shape Few-Shot, Not Zero-Shot, Binding in Tiny Transformers: A Fully-Enumerable Study

arXiv:2607. 04926v1 Announce Type: cross Abstract: How does the way information reaches a transformer -- as symbolic tokens, a clean per-factor "oracle" code, or an entangled perceptual vector -- shape whether it binds that information compositionally?

By Yoshiyuki Ootani

llms safety

arXiv AI

Jun 9

Phantom transitions in language model fine-tuning

arXiv:2606. 07559v1 Announce Type: cross Abstract: Fine-tuning a language model on contexts whose correct completion has a near-synonym competitor often fails silently.

By Vaibhav Prakash, Jayasri Dontabhaktuni

llms rag fine-tuning

arXiv AI

Jul 8

Transformers converge to invariant algorithmic cores

arXiv:2602. 22600v2 Announce Type: replace-cross Abstract: Training selects for behavior, not circuitry: many weight configurations can implement the same function.

By Joshua S. Schiffman

llms

arXiv AI

Jul 15

Attractor Geometry of Transformer Memory: From Conflict Arbitration to Confident Hallucination

arXiv:2605. 05686v3 Announce Type: replace Abstract: Language models draw on two knowledge sources: facts baked into weights (parametric memory, PM) and information in context (working memory, WM).

By Qiyao Liang, Risto Miikkulainen, Ila Fiete

llms fine-tuning safety

arXiv AI

Aug 5

The Ignition Is Real, and It Lives at the Readout: Latent composition, difficulty-clocked ignition, and the interface-constituted commit in a recurrent-depth reasoner

arXiv:2608. 03263v1 Announce Type: cross Abstract: We test whether the "compositional ignition" reported in latent-reasoning models is real computation, an instrument artifact, or inherited from verbal training data.

By Simon Lam-Muir

Related stories

Off-Axis, On Purpose: Where a Transformer Computes Concepts and Why it Does So

Input Pathways Shape Few-Shot, Not Zero-Shot, Binding in Tiny Transformers: A Fully-Enumerable Study

Phantom transitions in language model fine-tuning

Transformers converge to invariant algorithmic cores

Attractor Geometry of Transformer Memory: From Conflict Arbitration to Confident Hallucination

The Ignition Is Real, and It Lives at the Readout: Latent composition, difficulty-clocked ignition, and the interface-constituted commit in a recurrent-depth reasoner