arXiv AI
Unlocking Token Rewards via Training-Free Reward Attribution
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Sep 4
Gradients Know What Outcomes Don't: Unlocking Reinforcement Learning for LLM Reasoning with Gradient-Aligned Rewards
The paper introduces Gradient-Aligned Reward (GAR), a reinforcement learning technique that uses truncated backpropagation to generate a compact gradient vector for each rollout and compares it to an expert-anchor gradient via cosine similarity. This dense, reasoning-aware reward improves large language model chain-of-thought reasoning on math benchmarks and transfers to other tasks without domain‑specific data, while adding less than 9% computational overhead. GAR outperforms existing baselines such as GRPO on Qwen3-4B and Qwen3-8B models.
arXiv Machine Learning
Jun 4
Outcome-Grounded Advantage Reshaping for Fine-Grained Credit Assignment in Mathematical Reasoning
arXiv:2601. 07408v2 Announce Type: replace-cross Abstract: Group Relative Policy Optimization (GRPO) has emerged as a promising critic-free reinforcement learning paradigm for reasoning tasks.
arXiv AI
Sep 28
Stepwise Intrinsic Rewards for Reasoning in Large Language Models
The paper introduces Stepwise Marginal Information Gain (MIG), an intrinsic process reward that evaluates how each reasoning step of a large language model (LLM) or vision-language model (VLM) improves the likelihood of the reference answer. MIG rewards only new likelihood maxima, preventing duplicate credit, and is combined with outcome, format, and self‑distillation objectives to guide training. Experiments on eight benchmarks show that this method outperforms outcome‑only reinforcement learning and improves accuracy by up to 4.8 points over binary‑reward training, including a 12.6‑point gain on MathVerse and a 12.9‑point advantage on vision‑language transfer at 7B parameters.
arXiv AI
Sep 1
GMTS: Gradient Magnitude-based Token Selection Improves RLVR Training for LLM Reasoning
arXiv:2608.30632v1 Announce Type: cross Abstract: Reinforcement learning (RL), particularly RL with Verifiable Rewards (RLVR), has recently emerged as a central paradigm for enhancing large language...
Hugging Face Trending Papers
Jun 3
GRAIL: Gradient-Reweighted Advantages for Reinforcement Learning with Verifiable Rewards
Reinforcement learning with verifiable rewards (e. g.
arXiv AI
Sep 7
Extremely Sparse Supervision Incentivizes Reasoning Ability
The paper reports that in on‑policy distillation for large language models, reasoning performance can be improved by supervising only a tiny fraction of generated tokens—sometimes just one or two tokens per reasoning trajectory, about 0.05% of all tokens. This sparse supervision consistently matches or exceeds full‑token training across nine teacher‑student setups on mathematical reasoning, and is also validated on coding reasoning, Llama models, and PPO‑based reinforcement learning with verifiable reward. The findings suggest that effective post‑training does not require token‑intensive supervision and may align more closely with natural learning processes that focus on critical reasoning steps.