arXiv Machine Learning
Direction-Aware Token Credit Assignment with Reward-Model Gradients
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Oct 16
Right Answer, Wrong Tokens: Tail-Aware Credit Calibration for RLVR
The paper introduces Tail‑Aware Credit Calibration (TACO), a method that adjusts the positive credit given to tokens in reinforcement learning for large language models (LLMs) by evaluating each token’s rarity relative to the policy’s expectations. TACO reduces credit for high tail‑risk tokens while still allowing them to influence learning, thereby mitigating the positive‑credit contamination problem seen in GRPO‑style methods. Experiments on four LLMs across eight reasoning benchmarks demonstrate that TACO consistently outperforms GRPO baselines and improves training stability for long‑horizon RL tasks.
arXiv AI
Jul 10
When Implausible Tokens Get Reinforced: Tail-Aware Credit Calibration for LLM Reinforcement Learning
arXiv:2607. 07976v1 Announce Type: cross Abstract: Reinforcement learning (RL) has achieved remarkable success in enhancing the reasoning capabilities of large language models (LLMs).
arXiv AI
Jun 18
Learning from Own Solutions: Self-Conditioned Credit Assignment for Reinforcement Learning with Verifiable Rewards
arXiv:2606. 18810v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has driven substantial progress in training LLMs for reasoning tasks, but representative methods such as GRPO assign uniform credit across all tokens, wasting gradient on routine tokens while under-crediting pivotal reasoning steps.
arXiv AI
Jul 29
CoRT: Counterfactual Replay for Token-Level Rubric-Guided Policy Optimization
arXiv:2607. 25659v1 Announce Type: new Abstract: Rubric-based reinforcement learning enriches language model training by evaluating model outputs against explicit criteria.
Hugging Face Trending Papers
Jun 17
Learning from Own Solutions: Self-Conditioned Credit Assignment for Reinforcement Learning with Verifiable Rewards
Reinforcement learning with verifiable rewards (RLVR) has driven substantial progress in training LLMs for reasoning tasks, but representative methods such as GRPO assign uniform credit across all tokens, wasting gradient on routine tokens while under-crediting pivotal reasoning steps. Existing token-level credit assignment methods require resources beyond the model's own rollouts.
arXiv AI
Oct 1
Semifactual Credit-Augmented Policy Optimization
arXiv:2609.40360v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has improved the reasoning capabilities of large language models (LLMs), yet their predictions...