Fed-GRPO is a federated training framework for Large Language Models that extends Group Relative Policy Optimization (GRPO) to operate without sharing raw data. It introduces three reward‑signal‑driven mechanisms—signal‑weighted aggregation, global reward calibration, and adaptive sparse communication—to guide client updates based on reward statistics. Experiments on mathematical reasoning tasks show that Fed‑GRPO outperforms other federated methods, approaches centralized performance, and reduces communication by up to 32× while allowing extreme compression with minimal accuracy loss.
arXiv:2606. 03094v1 Announce Type: new Abstract: Recent advances in language models have established reinforcement learning as the primary paradigm for eliciting self-correction and long-chain reasoning.
arXiv:2610.09804v1 Announce Type: new Abstract: Reinforcement learning is now central to eliciting reasoning in large language models, while in the popular algorithm Group Relative Policy Optimizatio...
arXiv:2608. 01556v1 Announce Type: new Abstract: Large language models are increasingly aligned to human preferences via reward modeling, but user preference data are sensitive and often cannot be centralized.
ConsensusBench is a new dataset that supplies rule‑based process‑level signals for large language model reasoning. It identifies key intermediate conclusions—called Consensus Nodes—by filtering correct trajectories and clustering semantically equivalent statements. By incorporating a process reward derived from these nodes into GRPO‑style reinforcement learning, the authors create ConsensusPR, which reduces reward sparsity and improves performance on benchmarks such as AIME, GSM8K, and MATH‑500.
arXiv:2610.07212v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) trains a language model on problems that may themselves be confidential, and the trained model ca...
arXiv:2505. 09655v5 Announce Type: replace-cross Abstract: Post-training LLMs with Reinforcement Learning, specifically Group Relative Policy Optimization (GRPO), has emerged as a paradigm for enhancing mathematical reasoning.
arXiv:2605. 21125v2 Announce Type: replace Abstract: Group Relative Policy Optimization (GRPO), a prominent algorithm within the Reinforcement Learning from Verifiable Rewards (RLVR) framework, has achieved strong results in improving the reasoning capabilities of large language models (LLMs).
arXiv:2606. 16771v1 Announce Type: new Abstract: As LLMs advance, post-training reinforcement learning (RL) increasingly relies on multi-dimensional rewards to cultivate comprehensive capabilities.
arXiv:2608. 10499v1 Announce Type: cross Abstract: Personalized Federated Reinforcement Learning (PFRL) takes a decentralized approach to storing and accessing information based on past experiences while keeping each client's data private during the learning of each client's policy.
The paper introduces Fed‑LSVI, a federated online reinforcement learning algorithm that uses linear function approximation in episodic Markov decision processes. It achieves a regret bound of ≥O(√{Md^3H^4T}) while only exchanging compressed sufficient statistics, thereby meeting privacy constraints. The method reduces communication cost to logarithmic in the number of episodes, a marked improvement over previous approaches that required linear communication.
arXiv:2604. 00860v3 Announce Type: replace Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has become a central post-training paradigm for improving the reasoning capabilities of large language models.
arXiv:2604.22785v2 Announce Type: replace Abstract: We develop a unified treatment of credit assignment for RL training in multi-agent LLM systems. We show that observed reward alone cannot distingui...