arXiv AI October 9, 2026 By Pengxin Guo, Shuang Zeng, Zonggen Li, Weiying Zheng, Mengting Liu, Liangqiong Qu

Fed-GRPO: Reward-Signal-Driven Federated Group Relative Policy Optimization

Fed-GRPO is a federated training framework for Large Language Models that extends Group Relative Policy Optimization (GRPO) to operate without sharing raw data. It introduces three reward‑signal‑driven mechanisms—signal‑weighted aggregation, global reward calibration, and adaptive sparse communication—to guide client updates based on reward statistics. Experiments on mathematical reasoning tasks show that Fed‑GRPO outperforms other federated methods, approaches centralized performance, and reduces communication by up to 32× while allowing extreme compression with minimal accuracy loss.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning

Jun 3

FGRPO: Federated GRPO with Adaptive Aggregation on Non-IID Data

arXiv:2606. 03094v1 Announce Type: new Abstract: Recent advances in language models have established reinforcement learning as the primary paradigm for eliciting self-correction and long-chain reasoning.

By Pengyu Chen, Shaowei Li, Kai Wang, Yunsheng Yuan, Kai Han, Jun Luo, Feng Li

llms reinforcement-learning fine-tuning safety

arXiv Machine Learning

Oct 8

BoT-GRPO: Efficient Process-Reward RL for Reasoning via Bag-of-Token Aggregation

arXiv:2610.09804v1 Announce Type: new Abstract: Reinforcement learning is now central to eliciting reasoning in large language models, while in the popular algorithm Group Relative Policy Optimizatio...

By Yingxiang Yang, Weihang Xiao, Zhunxuan Wang, Joshua Flashner, Niresh Agarwal

llms reinforcement-learning multimodal

arXiv Machine Learning

Aug 4

Rethinking Personalized Reward Modeling for LLMs under Preference Heterogeneity via Group-Debiased Federated Learning

arXiv:2608. 01556v1 Announce Type: new Abstract: Large language models are increasingly aligned to human preferences via reward modeling, but user preference data are sensitive and often cannot be centralized.

By Seongyoon Kim, Boryeong Cho, Jihwan Oh, Seokhyun Chung, Se-Young Yun

llms reinforcement-learning fine-tuning

arXiv Computation and Language

Sep 7

ConsensusBench: Benchmark of Consensus Nodes for LLM Reasoning via Outcome Reward Densifying

ConsensusBench is a new dataset that supplies rule‑based process‑level signals for large language model reasoning. It identifies key intermediate conclusions—called Consensus Nodes—by filtering correct trajectories and clustering semantically equivalent statements. By incorporating a process reward derived from these nodes into GRPO‑style reinforcement learning, the authors create ConsensusPR, which reduces reward sparsity and improves performance on benchmarks such as AIME, GSM8K, and MATH‑500.

By Shi-Qi Yan, Chao-Hong Tan, Qian Chen, Wen Wang, Xiangang Li, Zhen-Hua Ling

llms reinforcement-learning efficiency benchmarks

arXiv Machine Learning

Oct 7

Reward-Driven Learning under Prompt-Level Differential Privacy

arXiv:2610.07212v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) trains a language model on problems that may themselves be confidential, and the trained model ca...

By Jiachen Zhao, Antonia Januszewicz, Taeho Jung

llms reinforcement-learning fine-tuning safety

arXiv Machine Learning

Jun 16

DRA-GRPO: Your GRPO Needs to Know Diverse Reasoning Paths for Mathematical Reasoning

arXiv:2505. 09655v5 Announce Type: replace-cross Abstract: Post-training LLMs with Reinforcement Learning, specifically Group Relative Policy Optimization (GRPO), has emerged as a paradigm for enhancing mathematical reasoning.

By Xiwen Chen, Wenhui Zhu, Peijie Qiu, Xuanzhao Dong, Hao Wang, Haiyu Wu, Huayu Li, Aristeidis Sotiras, Yalin Wang, Abolfazl Razi

llms reinforcement-learning benchmarks safety

One email a morning, machine-written

Related stories

FGRPO: Federated GRPO with Adaptive Aggregation on Non-IID Data

BoT-GRPO: Efficient Process-Reward RL for Reasoning via Bag-of-Token Aggregation

Rethinking Personalized Reward Modeling for LLMs under Preference Heterogeneity via Group-Debiased Federated Learning

ConsensusBench: Benchmark of Consensus Nodes for LLM Reasoning via Outcome Reward Densifying

Reward-Driven Learning under Prompt-Level Differential Privacy

DRA-GRPO: Your GRPO Needs to Know Diverse Reasoning Paths for Mathematical Reasoning