arXiv Machine Learning
BoT-GRPO: Efficient Process-Reward RL for Reasoning via Bag-of-Token Aggregation
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Jun 5
Selective-Advantage Entropy-Adaptive Horizon GRPO: Asymmetric Token-Level Discounting for Efficient Reinforcement Learning of Language Models
arXiv:2606. 05434v1 Announce Type: new Abstract: Group Relative Policy Optimisation (GRPO) has emerged as an effective reinforcement-learning algorithm for aligning language models on reasoning tasks, but it treats every token position and every sampled rollout symmetrically.
arXiv Machine Learning
1d ago
Hindsight-Privileged RLVR: Single-Rollout Token Credit Without Group Sampling
arXiv:2610.20497v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) trains language models from checked task outcomes, but assigning a terminal reward to individual...
arXiv AI
Aug 7
All-Quadrant Bounded Clipping GRPO: Closing the Unbounded Blind Spot for Stable and Generalizable Training
arXiv:2601. 03895v2 Announce Type: replace-cross Abstract: Group Relative Policy Optimization (GRPO) has emerged as a popular algorithm for reinforcement learning with large language models (LLMs).
arXiv Machine Learning
Jul 30
ReCo: Reweighting GRPO Against Distributional Concentration
arXiv:2607. 26862v1 Announce Type: new Abstract: Group Relative Policy Optimization (GRPO) has become a standard reinforcement learning method for post-training language models.
arXiv Computation and Language
Sep 7
ConsensusBench: Benchmark of Consensus Nodes for LLM Reasoning via Outcome Reward Densifying
ConsensusBench is a new dataset that supplies rule‑based process‑level signals for large language model reasoning. It identifies key intermediate conclusions—called Consensus Nodes—by filtering correct trajectories and clustering semantically equivalent statements. By incorporating a process reward derived from these nodes into GRPO‑style reinforcement learning, the authors create ConsensusPR, which reduces reward sparsity and improves performance on benchmarks such as AIME, GSM8K, and MATH‑500.
arXiv AI
Sep 11
Why Sample What You Can Enumerate? Exact Policy Optimization for Genomic Tool Selection
The paper critiques the common reinforcement‑learning approach of sampling tool subsets when the full set of tools is enumerable, showing that sampling leads to degraded policy estimates and increased reward sparsity in genomic reasoning tasks. It proposes Full‑Group Policy Optimization (FGPO), which evaluates every tool subset and precomputes rewards in a table, thereby eliminating the need for frozen‑reasoner calls during training. Experiments across five frozen reasoners and three genomic benchmarks demonstrate that FGPO consistently outperforms GRPO, improving average scores by 6.75 points and reducing the number of invoked tools per question.