← Back to all news
arXiv Machine Learning By Shijie Li, Yilin Gao, Tieyuan Chen, Yun Luo, Weiyao Lin, Hang Yu

GIFT: Gradient Informed Filtering for Distillation Augmented Reasoning RL

Read the original on arXiv Machine Learning →

GIFT (Gradient‑Informed Filtering for Distillation‑Augmented Reasoning RL) is a new method that resolves conflicts between reinforcement learning (RL) and distillation signals when training large language models for complex reasoning. It decomposes the distillation gradient into components parallel and orthogonal to the RL gradient, using the parallel part as a safety filter for conflict and the orthogonal part as a value measure for novel guidance. A progressive coverage‑annealing schedule then selectively applies token‑level guidance, gradually shifting from dense, filtered distillation to pure outcome‑grounded RL, yielding consistent performance gains—up to 9.5 % on challenging mathematical reasoning tasks—across multiple model scales.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Sep 7

Extremely Sparse Supervision Incentivizes Reasoning Ability

The paper reports that in on‑policy distillation for large language models, reasoning performance can be improved by supervising only a tiny fraction of generated tokens—sometimes just one or two tokens per reasoning trajectory, about 0.05% of all tokens. This sparse supervision consistently matches or exceeds full‑token training across nine teacher‑student setups on mathematical reasoning, and is also validated on coding reasoning, Llama models, and PPO‑based reinforcement learning with verifiable reward. The findings suggest that effective post‑training does not require token‑intensive supervision and may align more closely with natural learning processes that focus on critical reasoning steps.

By Zhishuai Liu, Xingzi Xu, Mehmet Saygin Seyfioglu, Pan Xu, Karim Bouyarmane
llmsreinforcement-learningefficiency
More like this →
arXiv AI
Jun 19

Reinforcement-aware Knowledge Distillation for LLM Reasoning

arXiv:2602. 22495v3 Announce Type: replace-cross Abstract: Reinforcement learning (RL) post-training has recently driven major gains in long chain-of-thought reasoning large language models (LLMs), but the high inference cost of such models motivates distillation into smaller students.

By Zhaoyang Zhang, Shuli Jiang, Yantao Shen, Yuting Zhang, Dhananjay Ram, Shuo Yang, Zhuowen Tu, Wei Xia, Stefano Soatto
llmsreinforcement-learningfine-tuningefficiencybenchmarks
More like this →
arXiv Machine Learning
Jul 22

H$^2$SD: Hybrid Hindsight Self-Distillation

arXiv:2607. 18955v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has substantially improved the reasoning capabilities of large language models on tasks such as mathematical reasoning and code generation.

By Qiye Cai, Yichuan Ma, Linyang Li, Peiji Li, Yongkang Chen, Qipeng Guo, Yicheng Zou, Tao Gui, Xiaocheng Feng, Bing Qin
llmsreinforcement-learningefficiencybenchmarks
More like this →