Hugging Face Trending Papers
GIFT: Gradient Informed Filtering for Distillation Augmented Reasoning RL
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Machine Learning
6d ago
GIFT: Gradient Informed Filtering for Distillation Augmented Reasoning RL
GIFT (Gradient‑Informed Filtering for Distillation‑Augmented Reasoning RL) is a new method that resolves conflicts between reinforcement learning (RL) and distillation signals when training large language models for complex reasoning. It decomposes the distillation gradient into components parallel and orthogonal to the RL gradient, using the parallel part as a safety filter for conflict and the orthogonal part as a value measure for novel guidance. A progressive coverage‑annealing schedule then selectively applies token‑level guidance, gradually shifting from dense, filtered distillation to pure outcome‑grounded RL, yielding consistent performance gains—up to 9.5 % on challenging mathematical reasoning tasks—across multiple model scales.
arXiv AI
Sep 7
Extremely Sparse Supervision Incentivizes Reasoning Ability
The paper reports that in on‑policy distillation for large language models, reasoning performance can be improved by supervising only a tiny fraction of generated tokens—sometimes just one or two tokens per reasoning trajectory, about 0.05% of all tokens. This sparse supervision consistently matches or exceeds full‑token training across nine teacher‑student setups on mathematical reasoning, and is also validated on coding reasoning, Llama models, and PPO‑based reinforcement learning with verifiable reward. The findings suggest that effective post‑training does not require token‑intensive supervision and may align more closely with natural learning processes that focus on critical reasoning steps.
arXiv Machine Learning
Jul 31
$\beta$-OPSD: Deriving with Policy Optimization, Training with Self-Distillation
arXiv:2607. 28582v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) is a promising approach to improve reasoning language models, but it remains brittle in practice: making it work reliably often requires substantial engineering effort.
arXiv AI
Jun 19
Reinforcement-aware Knowledge Distillation for LLM Reasoning
arXiv:2602. 22495v3 Announce Type: replace-cross Abstract: Reinforcement learning (RL) post-training has recently driven major gains in long chain-of-thought reasoning large language models (LLMs), but the high inference cost of such models motivates distillation into smaller students.
arXiv AI
Jun 18
Rethinking Reward Supervision: Rubric-Conditioned Self-Distillation
arXiv:2606. 19327v1 Announce Type: new Abstract: Post-training of reasoning language models is commonly driven by supervised distillation and reinforcement learning with verifiable rewards.
Hugging Face Trending Papers
Jun 17
Rethinking Reward Supervision: Rubric-Conditioned Self-Distillation
Post-training of reasoning language models is commonly driven by supervised distillation and reinforcement learning with verifiable rewards. Distillation often relies on chain-of-thought annotations that are expensive to obtain and may themselves be noisy, incomplete, or partially incorrect; even when the final solution is correct, an imperfect rationale can interfere with learning.