arXiv AI August 20, 2026 By Rahul Chowdhury, Timothy A Rupprecht, Senhao Cao, Jiahao Liu, Octavia Camps, David Bau, Pu Zhao, Yanzhi Wang

Mechanistic Interpretability of Structure-Aware Numerical Reasoning in LLaMA 3.1 8B

Read the original on arXiv AI →

The paper investigates how LLaMA 3.1‑8B models numerical sequence patterns, focusing on time‑series prediction. By designing a task that requires detecting structural cues—specifically first differences in a sequence—the authors show that the model performs well and internally computes and stores these differences. Probing and activation‑patching experiments reveal that LLaMA retrieves and applies the first‑difference via an induction‑like circuit, marking one of the first demonstrations of concept induction in large language models.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Aug 5

Uncovering Spontaneous Physics Representations in In-Context Learning

arXiv:2508. 12448v2 Announce Type: replace-cross Abstract: In-context learning (ICL) lets large language models (LLMs) solve new tasks from prompts alone, across an ever-widening range of domains, yet the mechanisms underlying this ability remain poorly understood.

By Yeongwoo Song, Jaeyong Bae, Dong-Kyum Kim, Hawoong Jeong

llms

arXiv Machine Learning

Jul 15

A Shared Subcircuit Lets LLMs Count Down Across Tasks

arXiv:2607. 12279v1 Announce Type: cross Abstract: Writing a sentence of exactly twelve words; ending a DNA sequence at the right codon; formatting an ASCII table.

By Jacob Dunefsky, Wes Gurnee, Emmanuel Ameisen

llms

arXiv Machine Learning

Aug 27

Towards A Unified Information Bottleneck Framework for Time Series Explanations

arXiv:2608. 25897v1 Announce Type: new Abstract: Explaining deep learning models operating on time series data is crucial in various applications that require transparent and interpretable insights into model behavior.

By Xu Zheng, Zichuan Liu, Zhuomin Chen, Mayur Akewar, Janki Bhimani, Jason Liu, Mo Sha, Jingchao Ni, Wei Cheng, Dongsheng Luo

benchmarks safety

arXiv AI

Jun 9

Mechanistic Data Attribution: Tracing the Training Origins of Interpretable LLM Units

arXiv:2601. 21996v2 Announce Type: replace-cross Abstract: While Mechanistic Interpretability has identified interpretable circuits in LLMs, their causal origins in training data remain elusive.

By Jianhui Chen, Yuzhang Luo, Liangming Pan

llms safety

Hugging Face Trending Papers

Jul 14

A Shared Subcircuit Lets LLMs Count Down Across Tasks

Writing a sentence of exactly twelve words; ending a DNA sequence at the right codon; formatting an ASCII table. These are all tasks that language models can do that requires tracking how many tokens remain before a target.

llms

Hugging Face Trending Papers

Aug 26

Towards A Unified Information Bottleneck Framework for Time Series Explanations

Explaining deep learning models operating on time series data is crucial in various applications that require transparent and interpretable insights into model behavior. {Existing explanation methods...

benchmarks safety

One email a morning, machine-written

Related stories

Uncovering Spontaneous Physics Representations in In-Context Learning

A Shared Subcircuit Lets LLMs Count Down Across Tasks

Towards A Unified Information Bottleneck Framework for Time Series Explanations

Mechanistic Data Attribution: Tracing the Training Origins of Interpretable LLM Units

A Shared Subcircuit Lets LLMs Count Down Across Tasks

Towards A Unified Information Bottleneck Framework for Time Series Explanations