← Back to all news
arXiv Computation and Language By Riku Kisako, Tatsuki Kuribayashi, Ryohei Sasano

Are Linguistic and Arithmetic Circuits Dissociated in LLMs?

Read the original on arXiv Computation and Language →

The paper investigates whether linguistic and arithmetic tasks in large language models (LLMs) rely on shared neural circuits. Using mechanistic interpretability techniques on five LLMs, the authors find that circuit similarity aligns primarily with task domain rather than input/output vocabularies or syntactic structure. This suggests that LLMs may employ largely separate neural circuits for language and arithmetic, indicating a modular organization of abilities.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv Computation and Language
Sep 4

How Much Do Circuits Tell Us? Measuring the Consistency and Specificity of Language Model Circuits

The study investigates the consistency and specificity of language model circuits across six tasks and five models, focusing on component-level (attention heads and MLP blocks) and neuron-level circuits. Component-level circuits are highly consistent and causally important but lack task specificity, as ablating a circuit for one task similarly harms performance on other tasks. Neuron-level circuits show higher task specificity but lower consistency, with overlap mainly between closely related tasks. The analysis of Llama‑3.2‑3B reveals that shared components are predominantly MLP blocks, while attention heads act as generic attention‑sink heads.

By Michael Li, Nishant Subramani
llmssafety
More like this →
arXiv AI
Sep 7

Shared circuits predict whether LLMs generalize across formats in arithmetic reasoning

The study investigates whether large language models (LLMs) can generalize arithmetic reasoning across different input formats, such as numeric and verbal expressions in English, Spanish, and Italian. By using attribution patching, the researchers identified the specific internal circuits each model employs for numeric versus verbal arithmetic tasks. They found that the degree of overlap between a model’s numeric circuit and its verbal circuit predicts how well the model generalizes to verbal formats, explaining variations in performance across languages and formats without requiring labeled data.

By Andrea Gregor de Varda, Sana Pandey, Pengrui Han, Jacob Andreas, Evelina Fedorenko
llms
More like this →
arXiv AI
Aug 17

Modular Cognitive Architecture Emerges in Large Language Models

arXiv:2608. 13567v1 Announce Type: new Abstract: The human brain exhibits a striking degree of functional specialization, with distinct networks supporting language, formal reasoning, reasoning about other minds, and reasoning about the physical world.

By Pengrui Han, Jacob Andreas, Evelina Fedorenko, Andrea Gregor de Varda
llms
More like this →