← Back to all news
arXiv AI By Andrea Gregor de Varda, Sana Pandey, Pengrui Han, Jacob Andreas, Evelina Fedorenko

Shared circuits predict whether LLMs generalize across formats in arithmetic reasoning

Read the original on arXiv AI →

The study investigates whether large language models (LLMs) can generalize arithmetic reasoning across different input formats, such as numeric and verbal expressions in English, Spanish, and Italian. By using attribution patching, the researchers identified the specific internal circuits each model employs for numeric versus verbal arithmetic tasks. They found that the degree of overlap between a model’s numeric circuit and its verbal circuit predicts how well the model generalizes to verbal formats, explaining variations in performance across languages and formats without requiring labeled data.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Computation and Language
Oct 21

Are Linguistic and Arithmetic Circuits Dissociated in LLMs?

The paper investigates whether linguistic and arithmetic tasks in large language models (LLMs) rely on shared neural circuits. Using mechanistic interpretability techniques on five LLMs, the authors find that circuit similarity aligns primarily with task domain rather than input/output vocabularies or syntactic structure. This suggests that LLMs may employ largely separate neural circuits for language and arithmetic, indicating a modular organization of abilities.

By Riku Kisako, Tatsuki Kuribayashi, Ryohei Sasano
llmssafety
More like this →
arXiv AI
Jul 15

Rethinking Reward Models for Multi-Domain Test-Time Scaling

arXiv:2510. 00492v3 Announce Type: replace Abstract: The reliability of large language models (LLMs) during test-time scaling is often assessed with \emph{external verifiers} or \emph{reward models} that distinguish correct reasoning from flawed logic.

By Dong Bok Lee, Seanie Lee, Sangwoo Park, Minki Kang, Jinheon Baek, Dongki Kim, Dominik Wagner, Jiongdao Jin, Heejun Lee, Tobias Bocklet, Jinyu Wang, Jingjing Fu, Sung Ju Hwang, Jiang Bian, Lei Song
llmsreinforcement-learning
More like this →
Hugging Face Trending Papers
Aug 13

Numeracy in Large Language Models: Fundamental Limitations and Paths to Improvement

Large language models (LLMs) achieve strong results on mathematical reasoning benchmarks yet remain unreliable on elementary numerical tasks, including magnitude comparison, large-integer arithmetic, fractions, and scientific notation. This survey examines basic numerical understanding as a capability distinct from high-level mathematical reasoning.

llmsragnlpfine-tuningbenchmarks
More like this →
arXiv AI
Sep 2

Do as I Say, Not as I Do: Instruction-Induction Conflict in LLMs

The paper investigates how large language models balance instruction-following with pattern completion when the two objectives conflict. By creating dialogues where a user instruction to act in a target way T is opposed by assistant turns that demonstrate a competing pattern P, the authors measure instruction-following rates across 13 models and 16 instructions over up to 50 turns. Results show wide variability (1%–99%) in instruction adherence, with robustness influenced by instruction content, output format, and chain-of-thought reasoning, but overall instruction-following remains brittle under induction pressure.

By Carolina Camassa, Derek Shiller
llmsbenchmarks
More like this →