arXiv AI September 15, 2026 By Quoc-Vinh Lai-Dang, Hyo-Sang Shin

HISPO: Hierarchical Importance-Sampling Policy Optimization with Entropy-Derived Segments

HISPO introduces a segment‑level policy‑optimization technique for reinforcement learning with verifiable rewards, creating entropy‑derived contiguous segments during rollout and applying clipped importance‑sampling correction at this granularity. It bridges the gap between token‑level and sequence‑level corrections, offering a middle‑ground approach for credit assignment in long‑form mathematical reasoning. Evaluated on Qwen3‑1.7B‑Base across six benchmarks, HISPO consistently outperforms or matches the strongest baselines in Pass@8 and Acc@8 metrics, notably improving AIME25 scores over GRPO and GSPO.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language

Sep 22

What are Key Factors for Updates in RL for LLM Reasoning?

arXiv:2606.22570v2 Announce Type: replace Abstract: Reinforcement Learning from Verifiable Rewards (RLVR) has emerged as a promising framework for enhancing the reasoning ability of large language mo...

By Peidong Wang, Demi Wang, Xufang Luo, Jiahang Xu, Xiaocui Yang, Shi Feng, Yuqing Yang, Dongsheng Li

llms reinforcement-learning benchmarks

arXiv Machine Learning

Jun 4

Outcome-Grounded Advantage Reshaping for Fine-Grained Credit Assignment in Mathematical Reasoning

arXiv:2601. 07408v2 Announce Type: replace-cross Abstract: Group Relative Policy Optimization (GRPO) has emerged as a promising critic-free reinforcement learning paradigm for reasoning tasks.

By Ziheng Li, Liu Kang, Feng Xiao, Luxi Xing, Qingyi Si, Zhuoran Li, Weikang Gong, Deqing Yang, Yanghua Xiao, Hongcheng Guo

llms reinforcement-learning benchmarks

arXiv AI

Jun 2

Revisiting Reinforcement Learning with Verifiable Rewards from a Contrastive Perspective

arXiv:2605. 12969v3 Announce Type: replace-cross Abstract: Group Relative Policy Optimization (GRPO) is one of the most widely adopted RLVR algorithms for post-training large language models on reasoning tasks.

By Feng Zhang, Xinhong Ma, Ziqiang Dong, Xi Leng, Jianfei Zhao, Xin Sun, Yang Yang, Guanjun Jiang

llms reinforcement-learning benchmarks

arXiv Machine Learning

Jun 25

Learning with a Single Rollout via Monte Carlo Pass@k Critic

arXiv:2606. 25451v1 Announce Type: new Abstract: Estimating token-level advantages in reinforcement learning (RL) for language models remains challenging because scaling up episodic experience collection is expensive.

By Fengdi Che, Yang Liu, Lei Yu, Meng Cao, Tong Che, Rupam Mahmood, Dale Schuurmans

llms reinforcement-learning benchmarks

arXiv Machine Learning

Jul 9

UP: Unbounded Positive Asymmetric Optimization for Breaking the Exploration-Stability Dilemma

arXiv:2607. 06987v1 Announce Type: new Abstract: Reinforcement learning (RL) has become the standard paradigm for enhancing the complex reasoning capabilities of large language models (LLMs).

By Chongyu Fan, Pengfei Liu, Jingjia Huang, Sijia Liu, Yi Lin

llms reinforcement-learning multimodal

arXiv AI

Aug 11

An Expectation-Maximization Perspective on Reinforcement Learning for LLM Reasoning

arXiv:2504. 18587v2 Announce Type: replace-cross Abstract: Reinforcement learning has emerged as a powerful approach for improving the reasoning capabilities of large language models, as demonstrated by systems such as OpenAI's O1~\cite{o1} and DeepSeek-R1~\cite{r1}.

By Tianbing Xu

llms reinforcement-learning safety

One email a morning, machine-written

Related stories

What are Key Factors for Updates in RL for LLM Reasoning?

Outcome-Grounded Advantage Reshaping for Fine-Grained Credit Assignment in Mathematical Reasoning

Revisiting Reinforcement Learning with Verifiable Rewards from a Contrastive Perspective

Learning with a Single Rollout via Monte Carlo Pass@k Critic

UP: Unbounded Positive Asymmetric Optimization for Breaking the Exploration-Stability Dilemma

An Expectation-Maximization Perspective on Reinforcement Learning for LLM Reasoning