arXiv AI June 18, 2026 By Laura Gomezjurado Gonzalez

The Long Delay to Arithmetic Generalization: When Learned Representations Outrun Behavior

arXiv:2604. 13082v2 Announce Type: replace-cross Abstract: Grokking in transformers trained on algorithmic tasks is characterized by a long delay between training-set fit and abrupt generalization, but the source of that delay remains poorly understood.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jul 24

Scaling Interpretable Transformers with Parity Bottleneck Layers

arXiv:2607. 20652v1 Announce Type: cross Abstract: Language models are thought to exhibit the phenomenon of superposition, representing many more features than dimensions in their residual streams.

By Andrew Mack, Kraig Yuheng Tou, Mark Henry, Zhengxun Wu, Lauren Greenspan

llms efficiency safety

arXiv AI

Jul 8

Transformers converge to invariant algorithmic cores

arXiv:2602. 22600v2 Announce Type: replace-cross Abstract: Training selects for behavior, not circuitry: many weight configurations can implement the same function.

By Joshua S. Schiffman

llms

arXiv Machine Learning

Sep 14

Breaking the Token Ceiling: Distilling Smaller, Stronger Byte Models

The paper investigates whether small models distilled from larger ones behave similarly when using byte versus token tokenization. It introduces two methods—Marginalize‑It (approximate) and End‑Of‑Token (exact)—to convert token logits to byte logits, and conducts a large‑scale study on decoder‑only dense transformers ranging from 1 billion to 1 trillion bytes of data. Results show that while token‑based models excel early, byte‑based models eventually surpass them with more compute, achieving higher performance ceilings, greater data efficiency, and lower logit storage costs.

By Kalyani Marathe, Artidoro Pagnoni, Tomasz Limisiewicz, Margaret Li, Mike Lewis, Luke Zettlemoyer, Srinivasan Iyer

llms nlp efficiency benchmarks

Hugging Face Trending Papers

Jul 9

Hidden Decoding at Scale: Latent Computation Scaling for Large Language Models

Scaling Large Language Models (LLMs) has been driven mainly by enlarging the Transformer backbone, but for an already-strong model this requires another round of costly pretraining. We study whether an existing backbone can keep improving by allocating more computation to each token while leaving the Transformer backbone fixed.

llms rag

arXiv Machine Learning

Jul 7

Structure-Specific Representational Priors Causally Control the Grokking Delay

arXiv:2607. 04333v1 Announce Type: new Abstract: Grokking -- generalization arriving long after training-set interpolation -- can be accelerated by structure-agnostic interventions: gradient filtering, weight-norm clamping, geometric penalties on hidden representations.

By Gunner Levi Howe

llms

arXiv Machine Learning

Jun 26

What Survives When You Compress a Recursive Reasoner for the Edge?

arXiv:2606. 26488v1 Announce Type: new Abstract: Recursive reasoning models can solve complex structured tasks with only a few million parameters by repeatedly updating a latent state.

By Pearse Jim, Steven Kolawole, Opegbemi Matthias Busoye, Glory Bagai, Virginia Smith

rag efficiency

One email a morning, machine-written

Related stories

Scaling Interpretable Transformers with Parity Bottleneck Layers

Transformers converge to invariant algorithmic cores

Breaking the Token Ceiling: Distilling Smaller, Stronger Byte Models

Hidden Decoding at Scale: Latent Computation Scaling for Large Language Models

Structure-Specific Representational Priors Causally Control the Grokking Delay

What Survives When You Compress a Recursive Reasoner for the Edge?