arXiv Computation and Language
PriFT: Guiding Supervised Fine-Tuning with Pretrained Priors
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv Machine Learning
Jun 9
PriFT: Prior-Support Guided Supervised Fine-Tuning
arXiv:2606. 09396v1 Announce Type: cross Abstract: Supervised fine-tuning (SFT) is an efficient approach for downstream task adaptation and often serves as the initialization stage for reinforcement learning (RL), but it can show weaker generalization than RL.
arXiv AI
Oct 12
What Should SFT Learn? Conditional Information Gain for Generalizable Post-Training
The paper introduces CIG‑SFT, a supervised fine‑tuning method that uses conditional information gain to evaluate how much each demonstrated token reduces uncertainty about the task. By comparing token likelihoods from a frozen reference model with and without task input, the method derives a KL‑regularized projection of the current policy, yielding a scalar‑weighted SFT objective whose gradients match the projected target. Experiments on mathematical reasoning and code‑generation benchmarks show that CIG‑SFT outperforms standard SFT and adaptive fine‑tuning baselines, especially in generalizing to challenging evaluation distributions, without requiring extra data or online rollouts.
arXiv AI
Jun 10
A Unifying Lens on Supervised Fine-Tuning Through Target Distribution Design
arXiv:2606. 11189v1 Announce Type: cross Abstract: Supervised fine-tuning (SFT) typically maximizes the likelihood of every token in a demonstrated trajectory.
arXiv AI
Aug 11
Proxy OPD: On-Policy Distillation with Transferable Relative Proxy Update
arXiv:2607. 11505v2 Announce Type: replace-cross Abstract: Post-training for large language models typically couples policy exploration with model optimization, hindering the reuse of high-reward behaviors from policy exploration.
arXiv AI
Jun 10
When RL Fails after SFT: Rejuvenating Model Plasticity for Robust SFT-to-RL Handoff
arXiv:2606. 09932v1 Announce Type: cross Abstract: Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (RL) has become a standard pipeline for Large Language Model (LLM) post-training.
arXiv Machine Learning
Sep 30
On-Policy Parameter Update Direction Underlies Generalization in LLM Post-Training
arXiv:2609.36659v1 Announce Type: new Abstract: The strong generalization performance of on-policy post-training paradigms has motivated studies of their parameter update behaviors. However, these st...