arXiv AI

Shared circuits predict whether LLMs generalize across formats in arithmetic reasoning

The study investigates whether large language models (LLMs) can generalize arithmetic reasoning across different input formats, such as numeric and verbal expressions in English, Spanish, and Italian. By using attribution patching, the researchers identified the specific internal circuits each model employs for numeric versus verbal arithmetic tasks. They found that the degree of overlap between a model’s numeric circuit and its verbal circuit predicts how well the model generalizes to verbal formats, explaining variations in performance across languages and formats without requiring labeled data.

arXiv Computation and Language

Oct 21

Are Linguistic and Arithmetic Circuits Dissociated in LLMs?

The paper investigates whether linguistic and arithmetic tasks in large language models (LLMs) rely on shared neural circuits. Using mechanistic interpretability techniques on five LLMs, the authors find that circuit similarity aligns primarily with task domain rather than input/output vocabularies or syntactic structure. This suggests that LLMs may employ largely separate neural circuits for language and arithmetic, indicating a modular organization of abilities.

By Riku Kisako, Tatsuki Kuribayashi, Ryohei Sasano

llms safety

arXiv AI

Aug 14

Numeracy in Large Language Models: Fundamental Limitations and Paths to Improvement

arXiv:2608. 13129v1 Announce Type: new Abstract: Large language models (LLMs) achieve strong results on mathematical reasoning benchmarks yet remain unreliable on elementary numerical tasks, including magnitude comparison, large-integer arithmetic, fractions, and scientific notation.

By Aoxin Ni

llms rag nlp fine-tuning benchmarks

arXiv AI

Jul 28

Masked Distillation: Internalizing the Chain-of-Thought in Language Models

arXiv:2607. 22629v1 Announce Type: new Abstract: Large Reasoning Models (LRMs) produce long, explicit chains of intermediate steps before generating a final answer at inference time.

By Durgesh Kalwar, Vardhan Palod, Subbarao Kambhampati

llms efficiency

arXiv AI

Jul 15

Rethinking Reward Models for Multi-Domain Test-Time Scaling

arXiv:2510. 00492v3 Announce Type: replace Abstract: The reliability of large language models (LLMs) during test-time scaling is often assessed with \emph{external verifiers} or \emph{reward models} that distinguish correct reasoning from flawed logic.

By Dong Bok Lee, Seanie Lee, Sangwoo Park, Minki Kang, Jinheon Baek, Dongki Kim, Dominik Wagner, Jiongdao Jin, Heejun Lee, Tobias Bocklet, Jinyu Wang, Jingjing Fu, Sung Ju Hwang, Jiang Bian, Lei Song

llms reinforcement-learning

Hugging Face Trending Papers

Aug 13

Numeracy in Large Language Models: Fundamental Limitations and Paths to Improvement

Large language models (LLMs) achieve strong results on mathematical reasoning benchmarks yet remain unreliable on elementary numerical tasks, including magnitude comparison, large-integer arithmetic, fractions, and scientific notation. This survey examines basic numerical understanding as a capability distinct from high-level mathematical reasoning.

llms rag nlp fine-tuning benchmarks

arXiv AI

Sep 2

Do as I Say, Not as I Do: Instruction-Induction Conflict in LLMs

The paper investigates how large language models balance instruction-following with pattern completion when the two objectives conflict. By creating dialogues where a user instruction to act in a target way T is opposed by assistant turns that demonstrate a competing pattern P, the authors measure instruction-following rates across 13 models and 16 instructions over up to 50 turns. Results show wide variability (1%–99%) in instruction adherence, with robustness influenced by instruction content, output format, and chain-of-thought reasoning, but overall instruction-following remains brittle under induction pressure.

By Carolina Camassa, Derek Shiller

llms benchmarks

arXiv Computation and Language

Oct 8

Bridge Routing Heads: Where Multilingual Multi-hop Reasoning Lives in LLMs

arXiv:2610.09733v1 Announce Type: new Abstract: Multilingual LLMs answer the same multi-hop reasoning question across languages, but we lack a mechanistic account of whether they share an internal ci...

By Seunghan Kim, Minyeong Choe, Hyunil Kim, Haehyun Cho

llms

arXiv Machine Learning

Sep 23

Rewired or Gated? How Instruction Tuning Shapes Knowledge-Conflict Circuits in LLMs

arXiv:2609.25602v1 Announce Type: new Abstract: In language models, the choice between believing the prompt and believing the weights is made by a handful of identifiable attention heads. Instruction...

By Shubham Santosh Pandere, Gautam Ranka, Ritika Varshney, Navya Deshmukh, Roushni Sareen, Roshan Kumar Singh

llms fine-tuning benchmarks safety

arXiv Machine Learning

Oct 5

Objects Without Morphisms: What LLMs for Mathematics Do Not Represent

arXiv:2610.03551v1 Announce Type: new Abstract: Large language models (LLMs) have reached expert-level performance on competition mathematics largely through the volume of search placed around them:...

By Yanli Wang, Suijin Wang, Xiaopeng Yuan, Haohan Wang

llms benchmarks

arXiv AI

Sep 2

(V)LMs generalize beyond surface co-occurrence: Evidence from cross-modal number agreement

arXiv:2609.00443v1 Announce Type: cross Abstract: Language models learn about grammatical number primarily from co-occurrence, and show frequency effects as a result---sometimes taken to indicate tha...

By Zach Studdiford, Kanishka Misra

llms rag multimodal

arXiv AI

Jun 3

Testing LLM Arithmetic Reasoning Generalization with Automatic Numeric-Remapping Attacks

arXiv:2606. 03606v1 Announce Type: cross Abstract: Large language models achieve strong performance on arithmetic reasoning benchmarks, and one common response to arithmetic brittleness is to delegate computation to code.

By Malia Barker, Bishal Lakha, Edoardo Serra, Francesco Gullo

llms benchmarks

arXiv AI

Jul 22

Structured Synthetic Reasoning Data for Arithmetic Fine-Tuning of Small Language Models

arXiv:2607. 18266v1 Announce Type: new Abstract: Small language models are attractive for local deployment, but they often struggle with multi-step arithmetic reasoning.

By Jake O'Grady, Effirul Ramlan

llms fine-tuning benchmarks

Related stories