arXiv Computation and Language

Can Dialects Be Steered Like Languages? Sparse Neurons and Distributed Directions in Arabic LLMs

The paper investigates how Arabic dialects are represented in large language models and whether they can be steered at inference time. By analyzing neuron-level sparsity and vector steering, the authors find that only a small fraction of neurons encode dialect-specific features, while distributed activation directions are more effective for steering. Vector steering can induce dialectal output from both dialectal and MSA prompts, whereas neuron steering works only when the prompt is already dialectal.

arXiv Computation and Language

Oct 9

Does Modern Standard Arabic (MSA) Dominate Arabic Dialects in LLMs? A Representation-Level Analysis

The study investigates whether Modern Standard Arabic (MSA) dominates internal representations in large language models (LLMs) when generating Arabic, even if prompted with dialectal forms. By extending a language-dominance framework to 26 Arabic varieties, the authors find no evidence that MSA serves as a dominant internal representation; instead, dialect representations are densely overlapping, with mutual information dropping sharply compared to more distinct languages. The strongest separability effects vary across layers and architectures, indicating that generation bias toward MSA does not necessarily reflect internal representational dominance.

By Abdu Sallouh, Nicholas Popovi\v{c}, Michael F\"{a}rber

llms safety

arXiv Computation and Language

Sep 22

Some Dialects Are More Equal Than Others: Non-Prestigious Arabic Dialectal Bias in LLMs

arXiv:2609.23955v1 Announce Type: new Abstract: Previous work on Egyptian Arabic in NLP has focused largely on the prestigious Cairene Egyptian Arabic (CEA) dialect, resulting in a lack of representa...

By Mai Mohamed Eida, Ryan Dolan, Paul de Nijs, Jonathan Dunn

llms nlp benchmarks safety

Hugging Face Trending Papers

Aug 6

Different Perturbations, Different Mechanisms: Understanding Continued Pre-training for Zero-Shot Dialect Robustness

Dialectal variation remains a major challenge for multilingual language models. Perturbation-based continued pre-training (CPT) has emerged as a promising approach to improving robustness, yet existing work largely evaluates individual perturbation strategies in isolation and provides limited insight into why they work.

llms safety

Hugging Face Trending Papers

Sep 21

Some Dialects Are More Equal Than Others: Non-Prestigious Arabic Dialectal Bias in LLMs

Previous work on Egyptian Arabic in NLP has focused largely on the prestigious Cairene Egyptian Arabic (CEA) dialect, resulting in a lack of representation for the less prestigious Sa'idi Egyptian Ara...

llms nlp benchmarks safety

arXiv AI

Jun 12

Language Model Circuits Are Sparse in the Neuron Basis

arXiv:2601. 22594v2 Announce Type: replace-cross Abstract: The high-level concepts that a neural network uses to perform computation need not be aligned to individual neurons (Smolensky, 1986).

By Aryaman Arora, Zhengxuan Wu, Jacob Steinhardt, Sarah Schwettmann

llms benchmarks safety

arXiv Machine Learning

Sep 10

Disentangling Steering Vectors

arXiv:2609.07037v1 Announce Type: new Abstract: Activation steering has emerged as a lightweight, inference-time approach to control the behavior of Large Language Models (LLMs). However, traditional...

By Takeru Hiramatsu, Kyohei Atarashi, Koh Takeuchi, Hisashi Kashima

llms

arXiv Machine Learning

Aug 4

LangFIR: Discovering Sparse Language-Specific Features from Monolingual Data for Language Steering

arXiv:2604. 03532v2 Announce Type: replace-cross Abstract: Large language models (LLMs) show strong multilingual capabilities, yet reliably controlling the language of their outputs remains difficult.

By Sing Hieng Wong, Hassan Sajjad, A. B. Siddique

llms benchmarks

arXiv AI

Jul 13

GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs

arXiv:2507. 18043v2 Announce Type: replace-cross Abstract: Inference-time steering methods offer a lightweight alternative to fine-tuning large language models (LLMs) and vision-language models (VLMs) by modifying internal activations at test time without updating model weights.

By Duy Nguyen, Archiki Prasad, Elias Stengel-Eskin, Mohit Bansal

llms fine-tuning multimodal safety

arXiv AI

Aug 14

Steering the Language Axis: From Linear Decodability to Causal Control

arXiv:2608. 12334v1 Announce Type: cross Abstract: Despite the impressive multilingual capabilities of Large Language Models, the latent dynamics dictating language selection remain poorly understood.

By Arnav Srivastav

llms benchmarks

arXiv Machine Learning

Sep 10

LLM Layers Immediately Correct Each Other

arXiv:2609.07876v1 Announce Type: cross Abstract: Recent methods in language model interpretability employ techniques such as sparse autoencoders to decompose residual stream contributions into linea...

By Arjun Patrawala, Jiahai Feng, Erik Jones, Jacob Steinhardt

llms safety

arXiv Machine Learning

Jul 2

Prototype Language Models

arXiv:2607. 00510v1 Announce Type: new Abstract: Knowing which training examples drive outputs is fundamental to auditing, correcting, and understanding language models, yet for modern LLMs this remains expensive, approximate, and largely post-hoc.

By Dan Ley, Giang Nguyen, Himabindu Lakkaraju, Julius Adebayo

llms fine-tuning

arXiv AI

Sep 15

Signatures of Steerability in Activation Space of Language Models

arXiv:2609.14151v1 Announce Type: cross Abstract: Steering language models using a set of contrastive representations has been a canonical and computationally efficient method for controlling model b...

By Prajjwal Bhattarai, Tuka Alhanai

llms safety

Related stories