arXiv Machine Learning August 19, 2026 By Sihan Wang, Jiayi Zhao, Qingyan Cao, Hongbo Yao, Lin Shu

FishBack: Pullback Fisher Geometry for Optimal Activation Steering in Transformers

Read the original on arXiv Machine Learning →

FishBack introduces a pullback Fisher geometry approach for activation steering in transformers, challenging the common Euclidean assumption of intermediate activation spaces. By deriving a closed‑form steering direction based on the Fisher information metric of the softmax layer, the method achieves target concept changes with minimal off‑target distortion, especially in early and middle layers. Experiments on GPT‑2 Small, Llama‑3‑8B, and Qwen3‑8B demonstrate significant reductions in off‑target KL divergence compared to existing steering baselines.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

llms

arXiv AI

Jul 8

Transformers converge to invariant algorithmic cores

arXiv:2602. 22600v2 Announce Type: replace-cross Abstract: Training selects for behavior, not circuitry: many weight configurations can implement the same function.

By Joshua S. Schiffman

llms

arXiv Machine Learning

Jun 9

Beyond Linear Activation Steering: Invertible Latent Transformations for Controlling LLM Behavior

arXiv:2606. 08454v1 Announce Type: new Abstract: Activation steering provides a lightweight inference-time mechanism for controlling large language models (LLMs) by modifying their internal activation vectors toward desired behaviors.

By Tuc Nguyen, Thai Le

llms diffusion benchmarks safety

arXiv AI

Jun 9

Phantom transitions in language model fine-tuning

arXiv:2606. 07559v1 Announce Type: cross Abstract: Fine-tuning a language model on contexts whose correct completion has a near-synonym competitor often fails silently.

By Vaibhav Prakash, Jayasri Dontabhaktuni

llms rag fine-tuning

arXiv AI

Jul 14

Conditional Optimal Bridge for Riemannian Activation Steering

arXiv:2607. 10517v1 Announce Type: cross Abstract: Activation steering offers a lightweight alternative to fine-tuning for controlling large language models at inference time.

By Seyed Arshan Dalili, Ajay Narayanan Sridhar, Vijaykrishnan Narayanan, Mehrdad Mahdavi

llms fine-tuning safety

arXiv AI

Jun 11

Neural FOXP2 -- Language Specific Neuron Steering for Targeted Language Improvement in LLMs

arXiv:2602. 00945v2 Announce Type: replace-cross Abstract: LLMs are multilingual by training, yet their lingua franca is often English, reflecting English language dominance in pretraining.

By Anusa Saha, Tanmay Joshi, Vinija Jain, Aman Chadha, Amitava Das

llms

arXiv AI

Jun 2

MENTIS: What Belief Changes Under Alignment? Measuring Multi-Scale Latent Torsion in Language Models

arXiv:2606. 01060v1 Announce Type: cross Abstract: Preference alignment has substantially improved the observable behavior of large language models, yet it remains unclear what alignment changes internally.

By Partha Pratim Saha, Samarth Raina, Mayur Parvatikar, Amit Dhanda, Vinija Jain, Aman Chadha, Amitava Das

llms safety

One email a morning, machine-written

Related stories

Transformers converge to invariant algorithmic cores

Beyond Linear Activation Steering: Invertible Latent Transformations for Controlling LLM Behavior

Phantom transitions in language model fine-tuning

Conditional Optimal Bridge for Riemannian Activation Steering

Neural FOXP2 -- Language Specific Neuron Steering for Targeted Language Improvement in LLMs

MENTIS: What Belief Changes Under Alignment? Measuring Multi-Scale Latent Torsion in Language Models