arXiv:2507. 18043v2 Announce Type: replace-cross Abstract: Inference-time steering methods offer a lightweight alternative to fine-tuning large language models (LLMs) and vision-language models (VLMs) by modifying internal activations at test time without updating model weights.
arXiv:2608. 02957v1 Announce Type: new Abstract: Steering vectors (SVs) are widely used to influence the expression of concepts (e.
Task-Level Natural Language Priors as Learning Signals for Low-Resource LLM Training proposes Prior-Guided Tuning (PGT), a training approach that treats natural-language priors as auxiliary learning signals rather than just input context. The method introduces Contrastive Prior Steering (CPS), which adds positive and negative prior-conditioned auxiliary losses while preserving the original supervised objective. Experiments on AmbiMath, Jigsaw, and MNLI/HANS demonstrate that CPS consistently outperforms plain and prompt fine-tuning, achieving high accuracy and significant gains with limited training data.
arXiv:2607. 29062v1 Announce Type: new Abstract: Model capabilities have improved in large part due to scaling chain of thought.
arXiv:2607. 25907v1 Announce Type: cross Abstract: Activation steering controls model behavior by editing internal activations at inference time.
arXiv:2607. 14111v1 Announce Type: cross Abstract: Can small language models detect and report on perturbations their own internal activations?
arXiv:2607. 04223v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) reduces but does not eliminate hallucination, and existing detectors return a single answer-level score that does not indicate which sentence is unsupported, or why.
arXiv:2609.37568v1 Announce Type: new Abstract: Audio-visual large language models (AVLLMs) have made remarkable progress in multimodal understanding and reasoning through interactions among visual,...
The paper introduces genesis steering, a method that derives a steering vector from contrastive examples in a model’s own fine‑tuning data, and proposes using membership inference to build proxy datasets when such data are unavailable. Two theoretical results are presented on optimal displacement to the decision boundary and reduced proxy‑estimation bias. Experiments on T"ulu‑3‑8B and Llama‑3.1‑8B‑Instruct show that safety data from T"ulu‑3‑8B’s training mixture produce stronger steering than external datasets, and that membership‑based proxy construction can achieve competitive performance.
Audio-visual large language models (AVLLMs) have made remarkable progress in multimodal understanding and reasoning through interactions among visual, auditory, and linguistic information. However, re...
IDEA is a training‑free, input‑dependent steering method for large language models that matches activations to cluster‑specific directions aligned with a target concept. It clusters positive and negative activation supports per attention head, solves an optimal‑matching problem to create a pool of cluster‑conditional directions, and selects the best match for each input at inference time. This approach preserves the input’s original representation while improving the truth × info rate on TruthfulQA by an average of 9.9% (up to 23.5%) over input‑independent baselines.
arXiv:2606. 00995v1 Announce Type: new Abstract: Subliminal learning refers to a student language model acquiring a teacher's traits (e.