arXiv Machine Learning

OrderGrad: Optimizing Beyond the Mean with Order-Statistic Policy Gradient Estimation

arXiv:2606. 06096v1 Announce Type: new Abstract: Policy-gradient methods usually optimize expected return, but many real world applications care about distributional properties of returns: tail risk, outlier robustness, or best-of-K discovery.

arXiv Machine Learning
Jun 5

On Advantage Estimates for Max@K Policy Gradients

arXiv:2606. 06080v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards is widely used for post-training reasoning models, but sparse outcome rewards make exploration difficult.

By Shota Takashiro, Soichiro Nishimori, Paavo Parmas, Yongmin Kim, Kohsei Matsutani, Gouki Minegishi, Yusuke Iwasawa, Takeshi Kojima, Yutaka Matsuo