arXiv AI
Jul 8
Transformers converge to invariant algorithmic cores
arXiv:2602. 22600v2 Announce Type: replace-cross Abstract: Training selects for behavior, not circuitry: many weight configurations can implement the same function.
FishBack: Pullback Fisher Geometry for Optimal Activation Steering in Transformers
arXiv:2605. 17231v2 Announce Type: replace Abstract: Activation steering has emerged as a lightweight approach for modifying language model behavior without parameter updates, yet existing methods remain brittle: unstable across layers and prone to disturbing behavior unrelated to the target concept.
Related stories
arXiv Machine Learning
Jun 9
Beyond Linear Activation Steering: Invertible Latent Transformations for Controlling LLM Behavior
arXiv:2606. 08454v1 Announce Type: new Abstract: Activation steering provides a lightweight inference-time mechanism for controlling large language models (LLMs) by modifying their internal activation vectors toward desired behaviors.
arXiv AI
Jun 9
Phantom transitions in language model fine-tuning
arXiv:2606. 07559v1 Announce Type: cross Abstract: Fine-tuning a language model on contexts whose correct completion has a near-synonym competitor often fails silently.
arXiv AI
Jul 14
Conditional Optimal Bridge for Riemannian Activation Steering
arXiv:2607. 10517v1 Announce Type: cross Abstract: Activation steering offers a lightweight alternative to fine-tuning for controlling large language models at inference time.
arXiv AI
Jun 11
Neural FOXP2 -- Language Specific Neuron Steering for Targeted Language Improvement in LLMs
arXiv:2602. 00945v2 Announce Type: replace-cross Abstract: LLMs are multilingual by training, yet their lingua franca is often English, reflecting English language dominance in pretraining.
arXiv AI
Jun 2
MENTIS: What Belief Changes Under Alignment? Measuring Multi-Scale Latent Torsion in Language Models
arXiv:2606. 01060v1 Announce Type: cross Abstract: Preference alignment has substantially improved the observable behavior of large language models, yet it remains unclear what alignment changes internally.
arXiv AI
Jul 13
GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs
arXiv:2507. 18043v2 Announce Type: replace-cross Abstract: Inference-time steering methods offer a lightweight alternative to fine-tuning large language models (LLMs) and vision-language models (VLMs) by modifying internal activations at test time without updating model weights.
arXiv AI
Jul 21
Phantom Transitions in Language Model Fine-Tuning: A Density-Matrix Analysis
arXiv:2606. 07559v2 Announce Type: replace-cross Abstract: Fine-tuning a language model often fails silently when its correct completion must outrank a near-synonym competitor.
arXiv Machine Learning
Jun 26
The Geometry of Updates: Fisher Alignment at Vocabulary Scale
arXiv:2606. 27242v1 Announce Type: new Abstract: Training-free source selection for LLM families with shared vocabularies arises in scientific string domains such as SMILES, protein, and genomic sequences, where candidate corpora share a tokenizer but differ in prediction targets.
arXiv AI
Jul 29
Minimizing Targeted Activations: Input-Only Suppression of Evaluation-Awareness Latents in Large Language Models
arXiv:2607. 25907v1 Announce Type: cross Abstract: Activation steering controls model behavior by editing internal activations at inference time.
arXiv AI
Jul 28
Hierarchical Grading in Large Language Models
arXiv:2607. 22757v1 Announce Type: cross Abstract: We introduce Graded Large Language Models (GLLMs), an algebraic framework that equips the representation space of a transformer with a grading and propagates the induced weighted scalar action through embeddings, self-attention, and the training objective.
arXiv AI
1d ago
Exact Attention Sensitivity and the Geometry of Transformer Stability
arXiv:2602. 18849v2 Announce Type: replace-cross Abstract: We develop a sensitivity analysis for transformer attention in a geometry aligned with tokenwise computation.