← Back to all news
arXiv Machine Learning By Jinghan Zhang, Fengran Mo, Tharindu Cyril Weerasooriya, Ruimin Dai, Xiaoyan Han, Yanjie Fu, Dakuo Wang, Kunpeng Liu

StaRPO: Stability-Augmented Reinforcement Policy Optimization

Read the original on arXiv Machine Learning →

StaRPO is a stability‑augmented reinforcement learning framework that enhances large language models’ reasoning by incorporating two lightweight stability metrics—Autocorrelation Function (ACF) for local step‑to‑step coherence and Path Efficiency (PE) for global goal‑directedness—into the optimization objective. By combining these stability rewards with task rewards, StaRPO provides process‑aware feedback that reduces logical inconsistencies, structural erraticism, and redundancy in generated responses. Experiments on four reasoning benchmarks demonstrate that StaRPO consistently outperforms baseline methods, improving both final‑answer accuracy and logical stability.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Oct 2

Your Language Model is Its Own Critic: Reinforcement Learning with Value Estimation from Actor's Internal States

The paper introduces POISE, a reinforcement learning algorithm that uses a model’s internal states as a value estimator to reduce variance in reinforcement learning with verifiable rewards (RLVR). By employing a lightweight probe that reads internal signals during the forward pass, POISE predicts baselines online and uses a cross‑rollout construction to keep gradients unbiased. Experiments on Qwen3‑4B and OLMo3‑7B‑Instruct‑DPO across six domains show POISE outperforms existing RLVR baselines, offering more stable training and a value model that generalizes across tasks and scales with the policy.

By Yunho Choi, Jongwon Lim, Woojin Ahn, Minjae Oh, Jeonghoon Shim, Yohan Jo
llmsreinforcement-learning
More like this →