arXiv Machine Learning July 2, 2026 By Dan Ley, Giang Nguyen, Himabindu Lakkaraju, Julius Adebayo

Prototype Language Models

Read the original on arXiv Machine Learning →

arXiv:2607. 00510v1 Announce Type: new Abstract: Knowing which training examples drive outputs is fundamental to auditing, correcting, and understanding language models, yet for modern LLMs this remains expensive, approximate, and largely post-hoc.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jun 2

Prototype Transformer: Towards Language Model Architectures Interpretable by Design

arXiv:2602. 11852v2 Announce Type: replace Abstract: While state-of-the-art language models (LMs) surpass most humans in certain domains, their reasoning remains largely opaque, reducing trust and increasing the risk of deception and hallucination.

By Yordan Yordanov, Matteo Forasassi, Bayar Menzat, Ruizhi Wang, Chang Qi, Markus Kaltenberger, Amine M'Charrak, Tommaso Salvatori, Thomas Lukasiewicz

llms benchmarks safety

arXiv Machine Learning

Sep 10

LLM Layers Immediately Correct Each Other

arXiv:2609.07876v1 Announce Type: cross Abstract: Recent methods in language model interpretability employ techniques such as sparse autoencoders to decompose residual stream contributions into linea...

By Arjun Patrawala, Jiahai Feng, Erik Jones, Jacob Steinhardt

llms safety

arXiv AI

Jun 12

Language Model Circuits Are Sparse in the Neuron Basis

arXiv:2601. 22594v2 Announce Type: replace-cross Abstract: The high-level concepts that a neural network uses to perform computation need not be aligned to individual neurons (Smolensky, 1986).

By Aryaman Arora, Zhengxuan Wu, Jacob Steinhardt, Sarah Schwettmann

llms benchmarks safety

arXiv AI

Jul 24

Scaling Interpretable Transformers with Parity Bottleneck Layers

arXiv:2607. 20652v1 Announce Type: cross Abstract: Language models are thought to exhibit the phenomenon of superposition, representing many more features than dimensions in their residual streams.

By Andrew Mack, Kraig Yuheng Tou, Mark Henry, Zhengxun Wu, Lauren Greenspan

llms efficiency safety

arXiv AI

2d ago

Closing the Loop: Practical Training Recipes for Looped Language Models

arXiv:2610.00673v1 Announce Type: cross Abstract: Looped language models increase effective depth by repeatedly applying a shared block of layers, but existing large-scale recipes require multi-stage...

By Andrei Marchenko, Viacheslav Bezrukov, Oleg Kashurin, Inessa Fedorova, Dmitry Bocharov, Yuliana Shakhvalieva, Maria Tikhonova, Valerii Ternovskii

llms benchmarks

arXiv AI

4d ago

On Trajectory-Aware Training for Masked Diffusion Language Models

arXiv:2609.37974v1 Announce Type: cross Abstract: Masked diffusion models (MDMs) generate text by unmasking several tokens per step, but they are trained and sampled under different conditions. The m...

By Manuel Madeira, Amitis Shidani, Alice Bizeul, Victor Turrisi, Louis B\'ethune, Bhavika Devnani, Dan Busbridge, Pierre Ablin, Jo\~ao Monteiro

llms diffusion fine-tuning safety

One email a morning, machine-written

Related stories

Prototype Transformer: Towards Language Model Architectures Interpretable by Design

LLM Layers Immediately Correct Each Other

Language Model Circuits Are Sparse in the Neuron Basis

Scaling Interpretable Transformers with Parity Bottleneck Layers

Closing the Loop: Practical Training Recipes for Looped Language Models

On Trajectory-Aware Training for Masked Diffusion Language Models