← Back to all news

Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

6,602 stories · RSS feed

arXiv Machine Learning
17h ago

PerSPL: Personalized Semi-supervised Preference Learning

PerSPL (Personalized Semi-supervised Preference Learning) is a backbone‑agnostic framework that stabilizes cross‑prompt score coordinates using pair‑mean score centering, enabling the conversion of unlabeled responses into user‑specific training signals. It addresses the scarcity of user‑specific preference comparisons by leveraging large pools of unlabeled responses, which are otherwise unused by existing personalized reward models. Across four benchmarks—PersonalLLM, HelpSteer2, Reddit TLDR, and Chatbot Arena—PerSPL achieves the highest mean accuracy for both seen and unseen users, demonstrating its effectiveness in summarization and open‑domain chatbot interaction settings.

By Xiaoan Xu, Tingzhou Wei, Juncheng Dong, Korel Gundem, Ethan X. Fang, Zhengling Qi
nlpreinforcement-learningbenchmarks
More like this →
arXiv Machine Learning
17h ago

Remote Rowhammer Attack using Adversarial Observations on Federated Learning Clients

The paper demonstrates a novel remote Rowhammer attack that targets federated learning (FL) servers by manipulating client sensor observations to induce repetitive clustered memory updates. Using a reinforcement learning attacker, the authors show that they can achieve approximately 70% repeated update rate in FL systems with sparse updates, leading to bit‑flips in the server’s DRAM during tasks such as automatic speech recognition and CIFAR‑10 classification. This work highlights a previously underexplored vulnerability where client‑side adversarial behavior can compromise server memory integrity in FL environments.

By Jinsheng Yuan, Yuhang Hao, Weisi Guo, Yun Wu, Chongyan Gu
agentsreinforcement-learningmultimodalsafety
More like this →
arXiv Machine Learning
17h ago

EPIC: Rethinking Trust Regions for Autoregressive LLM Policies via Exact-KL Prefix Importance-Ratio Clipping

EPIC introduces a new PPO-style clipping method that operates at the prefix level rather than token level, using the exact reverse-KL coordinate to regulate policy drift during autoregressive generation. By reformulating generation as a finite-horizon Markov decision process, the authors derive a performance-difference identity that identifies the cumulative likelihood ratio of the generated prefix as the exact importance weight for each policy-improvement step. Experiments on off‑policy reinforcement learning with a sparse mixture‑of‑experts language model demonstrate that EPIC consistently outperforms standard and sequence‑level clipping baselines on mathematical‑reasoning benchmarks while remaining competitive with routing‑replay alternatives.

By Chengxi Li, Hanlin Wu, Zhicheng Cai, Huan-ang Gao, Wei-Ying Ma, Jingjing Liu, Hao Zhou
llmsreinforcement-learningbenchmarks
More like this →
arXiv Machine Learning
17h ago

Dissimilarity-Based Markov Decision Processes for Stabilizing State Distributions

arXiv:2610.29024v1 Announce Type: new Abstract: Reinforcement learning for stochastic systems typically optimizes expected cumulative rewards, while reliable and repeatable behavior also depends on h...

By Xun Shen, Yuepeng Wang, Daniyal Sajedi-Hosseini, Akifumi Wachi, Ken Kawano, Yoshihiko Fujisawa, Mehrshad Sadria, S\'ebastien Gros, Katsuki Fujisawa
reinforcement-learning
More like this →