The paper "Performance Foundations of Parallel & Distributed Reasoning Language Models" examines how reinforcement learning with verifiable rewards (RLVR) and similar post‑training methods improve reasoning in large language models, yet demand massive computational resources. It provides a compute‑centric analysis of key RL frameworks such as PPO and GRPO, and introduces a taxonomy of intra‑ and inter‑model parallelism strategies—including data, tensor, pipeline, sequence, context, expert, disaggregated placement, stage fusion, hybrid parallelism, and asynchronous execution—to address the parallel and distributed systems challenges of training reasoning language models. The authors also analyze existing RLM frameworks, offering practical guidelines and outlining open research directions for building scalable, fast, and cost‑effective RLMs.
By Maciej Besta, Leonard Schmidt, Lara Nonino, Robert Gerstenberger, Pierre Pang, Patrik Okanovic, Ales Kubicek, Tiancheng Chen, Baraq Lipshitz, Torsten Hoefler
arXiv:2507. 04136v2 Announce Type: replace Abstract: This survey offers a comprehensive foundation on the integration of RL with language models, highlighting prominent algorithms such as Proximal Policy Optimization (PPO), Q-Learning, and Actor-Critic methods.
By Saksham Sahai Srivastava, Vaneet Aggarwal
Parason is a new framework that discovers and exploits both subtask and trial parallelism in large language model (LLM) reasoning. By converting sequential reasoning traces into structured parallel trajectories and training with Parallelism-Aware Group Relative Policy Optimization, it balances accuracy, latency, and parallelism. Experiments on mathematical reasoning benchmarks such as AIME24 and AIME25 show that Parason achieves an average acceleration of about 1.7× while maintaining competitive accuracy.
By Zhengyang Zhang, Zijian Zhang, Jiaxuan Gao, Shusheng Xu, Yi Wu, Song Han, Ligeng Zhu
arXiv:2512. 07843v2 Announce Type: replace-cross Abstract: Scaling inference-time computation has enabled Large Language Models (LLMs) to achieve strong reasoning performance, but their inherently sequential decoding incurs substantial latency, motivating parallelization of the generation process.
By Long Lian, Sida Wang, Felix Juefei-Xu, Tsu-Jui Fu, Xiuyu Li, Adam Yala, Trevor Darrell, Alane Suhr, Yuandong Tian, Xi Victoria Lin
arXiv:2603.00889v2 Announce Type: replace-cross
Abstract: Large Language Models (LLMs) have exhibited remarkable reasoning capabilities, largely enabled by supervised fine-tuning (SFT)- and reinforce...
By Xinyu Zhu, Yihao Feng, Yanchao Sun, Xianzhi Du, Pingzhi Li, Olli Saarikivi, Yun Zhu, Yu Meng
The paper investigates how to apply Reinforcement Learning with Verifiable Rewards (RLVR) to large language models across multiple domains. It compares two training paradigms—mixed multi-task RLVR and separate RLVR followed by model merging—using tasks such as math, coding, science, instruction following, and agent. Experiments show that RLVR across domains causes minimal interference and that reasoning-intensive domains can synergize, with insights drawn from information constraints, prediction behavior, and self-verification.
By Haoqing Wang, Xiang Long, Ziheng Li, Yilong Xu, Tingguang Li, Yehui Tang