The paper introduces Minimally Invasive Steering Vector Optimization (MISVO), a method that adjusts a frozen language model’s final hidden states by adding vectors to steer outputs toward a test‑time reward while penalizing changes using the local KL geometry of the token distribution. MISVO derives an analytic Fisher term and a suffix score‑function term, showing that the suffix term is second‑order and that Fisher surrogates match the full KL gradient to first order. Experiments on preference and code‑generation tasks with 1B–14B parameter models demonstrate that MISVO achieves the highest mean reward in most settings while maintaining diversity and coherence comparable to Best‑of‑N.
arXiv:2605. 05983v2 Announce Type: replace Abstract: Recently, steering vectors (SVs) have emerged as an effective and lightweight approach to steer behaviors of large language models (LLMs), among which fine-tuned SVs are more effective than optimization-free ones.
arXiv:2606. 11599v1 Announce Type: cross Abstract: Activation steering offers a lightweight approach to control language models' behavior at inference time, but whether it succeeds or fails heavily depends on the prompt, concept, model, and steering configuration.
arXiv:2606. 09635v1 Announce Type: cross Abstract: Ensuring the reliability of Large Language Models (LLMs) under distribution drift requires inference-time adaptation.
arXiv:2607. 05615v1 Announce Type: new Abstract: Activation steering via sparse autoencoders (SAEs) enables behavioral control of large language models without task-specific fine-tuning, but standard methods apply the steering signal at every generated token, incurring constant per-token perturbation that risks degrading fluency.
Activation steering offers a lightweight approach to control language models' behavior at inference time, but whether it succeeds or fails heavily depends on the prompt, concept, model, and steering configuration. Finding the regime and boundaries of successful steering typically requires expensive grid searches and post-hoc evaluation of full autoregressive rollouts.
MetaSteer is a new method for steering large language models that learns nonlinear, context-dependent interventions applied to attention projection matrices. Unlike traditional linear, context-independent techniques, MetaSteer adapts its effects based on the input, requiring no linear concept-geometry assumption. Trained once on a pooled preference corpus, it transfers zero‑shot to unseen concepts and out‑of‑distribution contexts, matching or surpassing strong task‑specific baselines on multiple benchmarks and model families.
arXiv:2606. 15092v1 Announce Type: new Abstract: Activation steering has emerged as a key methodology for controlling the behavior of large language models (LLMs).
arXiv:2605. 14746v2 Announce Type: replace Abstract: While large language models (LLMs) are trained to align with human values, their generations may still violate safety constraints.
arXiv:2507. 18043v2 Announce Type: replace-cross Abstract: Inference-time steering methods offer a lightweight alternative to fine-tuning large language models (LLMs) and vision-language models (VLMs) by modifying internal activations at test time without updating model weights.
arXiv:2609.36202v1 Announce Type: cross Abstract: Gradient-based reward guidance provides a flexible way to use downstream reward models to control masked diffusion language models at inference time....
arXiv:2608. 05813v1 Announce Type: new Abstract: Personalizing language models (LMs) to individual user preferences is essential for aligning responses with diverse goals and backgrounds.
arXiv:2607. 23153v1 Announce Type: cross Abstract: Recent work has shown that large language models (LLMs) can iteratively improve their outputs by incorporating generated samples and their corresponding evaluation scores as in-context examples.