arXiv Computation and Language November 4, 2026 By Hirofumi Morise

Information-Theoretic Design of Sparse Autoencoder-Based Steering

Read the original on arXiv Computation and Language →

The paper introduces an information‑theoretic framework for designing sparse autoencoder‑based steering interventions in large language models. It uses the Jensen–Shannon divergence between empirical feature‑firing distributions to quantify how much a state deviates from a target distribution and to construct steering vectors that guide hidden states toward desired firing patterns. Experiments demonstrate the method’s effectiveness in question‑mode generation, show structured downstream effects of local interventions, and apply the framework to safety‑related feature analysis.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Machine Learning

Jun 11

When is Your LLM Steerable?

arXiv:2606. 11599v1 Announce Type: cross Abstract: Activation steering offers a lightweight approach to control language models' behavior at inference time, but whether it succeeds or fails heavily depends on the prompt, concept, model, and steering configuration.

By Chenrui Fan, Yize Cheng, Ming Li, Soheil Feizi, Tianyi Zhou

llms

arXiv Machine Learning

Aug 4

LangFIR: Discovering Sparse Language-Specific Features from Monolingual Data for Language Steering

arXiv:2604. 03532v2 Announce Type: replace-cross Abstract: Large language models (LLMs) show strong multilingual capabilities, yet reliably controlling the language of their outputs remains difficult.

By Sing Hieng Wong, Hassan Sajjad, A. B. Siddique

llms benchmarks

arXiv AI

Aug 12

SAE-StatSteer: Statistical Consensus Feature Selection for Optimization-Free Activation Steering of Large Language Models

arXiv:2607. 19364v2 Announce Type: replace Abstract: Activation steering adds a residual-stream direction at inference time, providing lightweight behavioral control without fine-tuning.

By Oshayer Siddique, J. M Areeb Uzair Alam, Md Jobayer Rahman Rafy, Syed Rifat Raiyan, Hasan Mahmud, Md Kamrul Hasan

llms fine-tuning

Hugging Face Trending Papers

Jun 10

When is Your LLM Steerable?

Activation steering offers a lightweight approach to control language models' behavior at inference time, but whether it succeeds or fails heavily depends on the prompt, concept, model, and steering configuration. Finding the regime and boundaries of successful steering typically requires expensive grid searches and post-hoc evaluation of full autoregressive rollouts.

llms

arXiv AI

Jul 13

GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs

arXiv:2507. 18043v2 Announce Type: replace-cross Abstract: Inference-time steering methods offer a lightweight alternative to fine-tuning large language models (LLMs) and vision-language models (VLMs) by modifying internal activations at test time without updating model weights.

By Duy Nguyen, Archiki Prasad, Elias Stengel-Eskin, Mohit Bansal

llms fine-tuning multimodal safety

arXiv Machine Learning

Jun 25

How Does the Pretraining Distribution Shape In-Context Learning? A Fundamental Trade-Off

arXiv:2510. 01163v2 Announce Type: replace Abstract: The factors driving the performance of in-context learning (ICL) in large language models (LLMs) remain poorly understood despite ICL's surprising effectiveness, enabling models to adapt to new tasks from only a handful of examples.

By Wa\"iss Azizian, Ali Hasan

llms

One email a morning, machine-written

Related stories

When is Your LLM Steerable?

LangFIR: Discovering Sparse Language-Specific Features from Monolingual Data for Language Steering

SAE-StatSteer: Statistical Consensus Feature Selection for Optimization-Free Activation Steering of Large Language Models

When is Your LLM Steerable?

GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs

How Does the Pretraining Distribution Shape In-Context Learning? A Fundamental Trade-Off