← Back to all news
arXiv Machine Learning By Tiejin Chen, Maunil Vyas, Huaiyuan Yao, Hua Wei

Steering Vectors as a Training Signal in LLM Post-Training

Read the original on arXiv Machine Learning →

Steering vectors (SVs) are residual-stream directions that can shift LLM behavior when added to hidden states during inference. This study investigates whether the same SVs can guide parameter updates during post‑training, exploring supervised learning (SFT and distillation) and reinforcement learning with verifiable rewards (RLVR). The authors find that SV injection improves performance over plain SFT and distillation, achieving up to a 12% gain in certain distillation settings, and that applying a correctness‑contrast direction as advantage repair in RL recovers learning signals for held‑out math benchmarks. They also identify failure boundaries related to code generation, cross‑family layer transfer, and inference‑time application on SV‑trained checkpoints.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Sep 4

Sequential Beats Joint: On the Interplay between On-Policy Distillation and RLVR

The paper investigates how on-policy distillation (OPD) and reinforcement learning with verifiable rewards (RLVR) can be combined for post‑training reasoning in large language models. It shows that a two‑stage approach—first applying OPD, then RL—outperforms single‑signal methods and other joint baselines on logic and math reasoning benchmarks. The authors explain this advantage through pass@k analysis, learning dynamics, and parameter updates, concluding that OPD expands solution coverage while RL sharpens performance within that support, and that the OPD validation score is the key trigger for switching to RL.

By Boyan Li, Bingsen Chen, Chenghao Yang, Ping Nie, Chen Zhao, Xi Ye
llmsreinforcement-learningfine-tuningefficiencybenchmarks
More like this →