arXiv Machine Learning
PACT: From Credit Assignment to Critic Alignment
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Jun 19
VIMPO: Value-Implicit Policy Optimization for LLMs
arXiv:2606. 20008v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards has become a central tool for improving the reasoning ability of large language models, but current methods face a trade-off between simplicity and credit assignment.
arXiv AI
Jun 18
Learning from Own Solutions: Self-Conditioned Credit Assignment for Reinforcement Learning with Verifiable Rewards
arXiv:2606. 18810v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has driven substantial progress in training LLMs for reasoning tasks, but representative methods such as GRPO assign uniform credit across all tokens, wasting gradient on routine tokens while under-crediting pivotal reasoning steps.
arXiv AI
Jul 29
CoRT: Counterfactual Replay for Token-Level Rubric-Guided Policy Optimization
arXiv:2607. 25659v1 Announce Type: new Abstract: Rubric-based reinforcement learning enriches language model training by evaluating model outputs against explicit criteria.
arXiv Machine Learning
Jun 25
Learning with a Single Rollout via Monte Carlo Pass@k Critic
arXiv:2606. 25451v1 Announce Type: new Abstract: Estimating token-level advantages in reinforcement learning (RL) for language models remains challenging because scaling up episodic experience collection is expensive.
arXiv AI
Aug 25
Let Credit Follow Computation: Architecture-Aware Credit Transport for Large Language Model Reinforcement Learning
arXiv:2608.21501v1 Announce Type: new Abstract: Credit assignment in large-language-model reinforcement learning (LLM RL) can be separated into three objects: evidence about success, a transport oper...
arXiv Machine Learning
Aug 4
Start Classifying: Categorical Critics for LLM Reinforcement Learning
arXiv:2608. 02181v1 Announce Type: new Abstract: Proximal Policy Optimization (PPO) for large language models typically trains its critic by mean-squared-error (MSE) regression on scalar value targets.