Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

7,004 stories · RSS feed

arXiv Machine Learning

Jul 24

Robust Asynchronous Q-Learning under Reward and State Corruption via Batching

arXiv:2607. 20822v1 Announce Type: new Abstract: Motivated by reinforcement learning in harsh environments, we consider the problem of learning an optimal policy subject to adversarially corrupted feedback.

By Sreejeet Maity, Aritra Mitra

reinforcement-learning

arXiv AI

Jul 24

ARCO: Adaptive Rubrics with Co-Evolution for Multi-Step LLM-Based Agents

arXiv:2606. 21262v2 Announce Type: replace Abstract: Reinforcement learning for multi-step LLM agents often relies on scalar rewards that indicate success but cannot explain why a trajectory is good or bad.

By Zihang Tian, Jingsen Zhang, Rui Li, Xiaohe Bo, Yuanzi Li, Xu Chen

llms agents reinforcement-learning safety

arXiv AI

Jul 24

When RLVR Shrinks the Reasoning Boundary: Diagnosing Pass@k Inversion

arXiv:2607. 20543v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) can improve one-sample accuracy while making a model worse under repeated sampling.

By Todd Zhou

agents reinforcement-learning multimodal benchmarks

arXiv Machine Learning

Jul 24

Exploiting Exogenous Structure for Sample-Efficient Reinforcement Learning

arXiv:2409. 14557v4 Announce Type: replace-cross Abstract: We study a structured class of Markov Decision Processes, known as Exo-MDPs, in which the state space is partitioned into exogenous and endogenous components.

By Jia Wan, Sean R. Sinclair, Devavrat Shah, Martin J. Wainwright

agents reinforcement-learning

arXiv Machine Learning

Jul 24

Perspective Latents as an Architectural Condition for Causal Emergence in Active Inference Agents

arXiv:2607. 20708v1 Announce Type: new Abstract: A recent line of work measures causal emergence in reinforcement learning agents through Integrated Information Decomposition, reporting that $\Phi_r$ grows with training and tracks reward improvement.

By Hongju Pae

agents reinforcement-learning

arXiv AI

Jul 24

Adaptive Multi-Horizon Reinforcement Learning

arXiv:2607. 20656v1 Announce Type: cross Abstract: Effective decision-making in complex and changing environments requires balancing short-term and long-term consequences.

By Manoosh Samiei, Doina Precup, Paul Masset

agents reinforcement-learning

arXiv AI

Jul 24

Beyond SBDD: Geometric Deep Learning in Polypharmacology and Multi-target Drug Design

arXiv:2607. 20550v1 Announce Type: cross Abstract: The traditional "one drug, one target" paradigm of structure-based drug design (SBDD) frequently proves inadequate for treating multifactorial diseases such as cancer and neurodegenerative disorders, owing to compensatory signaling pathways and the emergence of drug resistance.

By Tianming Han, Zhijie Pan, Wenchi Ge, Qi Zhao

rag diffusion reinforcement-learning multimodal benchmarks

arXiv Machine Learning

Jul 24

How Many Bits Can an Adapter Write? Measuring the Capacity and Memorization of Parameter-Efficient Fine-Tuning

arXiv:2607. 21351v1 Announce Type: new Abstract: A LoRA adapter is a few megabytes that almost everyone treats as a skill rather than a record of the data behind it.

By Kaizhen Tan, Heqing Du, Yang Feng

reinforcement-learning fine-tuning safety

arXiv AI

Jul 24

Training Large Language Models for Self-Explanation Faithfulness

arXiv:2607. 21090v1 Announce Type: cross Abstract: We propose a Reinforcement Learning (RL) method to directly optimize the faithfulness of self-explanations - the extent to which a model's generated reasoning accurately reflects its internal decision-making process.

By Yeoktatt Cheah, Mar\'ia P\'erez-Ortiz, Noah Y. Siegel, Oana-Maria Camburu

llms reinforcement-learning fine-tuning safety

arXiv Machine Learning

Jul 24

Error Amplification Limits ANN-to-SNN Conversion in Continuous Control

arXiv:2601. 21778v3 Announce Type: replace-cross Abstract: Spiking Neural Networks (SNNs) can achieve competitive performance by converting already existing well-trained Artificial Neural Networks (ANNs), avoiding further costly training.

By Zijie Xu, Zihan Huang, Yiting Dong, Kang Chen, Wenxuan Liu, Zhaofei Yu

reinforcement-learning benchmarks

arXiv AI

Jul 24

CMI-Mem: Toward Generalizable Long-Term Memory Management via CMI-Augmented Reinforcement Learning

arXiv:2607. 20553v1 Announce Type: new Abstract: Memory Manager models are pivotal in agent systems.

By Yubo Wang, Qiuyu Zhao, Zenghui Sun, Shichao Dong, Jinsong Lan, Xiaoyong Zhu, Haoyang Li, Bo Zheng, Lei Chen

llms agents reinforcement-learning

arXiv AI

Jul 24

SalesLoop: Reinforcement Learning from Performance Feedback for Sales Lead Ranking

arXiv:2607. 20655v1 Announce Type: cross Abstract: Lead ranking in Customer Relationship Management (CRM) systems faces a persistent challenge: models achieving high offline accuracy often underperform in production.

By Chenyu Zhang

reinforcement-learning

arXiv AI

Jul 24

Artificial Epanorthosis: Why large language models overuse a classical rhetorical figure, and how to mitigate it

arXiv:2607. 21498v1 Announce Type: cross Abstract: A rhetorical figure that Cicero and Quintilian catalogued two thousand years ago reappears, systematically, in the text of large language models: epanorthosis, the self-correction of the specimen {\guillemotleft}This is not a course.

By Federico Boggia

llms reinforcement-learning fine-tuning

arXiv Machine Learning

Jul 24

The Weight of Silence: A Causal Case for Weights Over the Scratchpad in Latent Chess Reasoning

arXiv:2607. 20952v1 Announce Type: new Abstract: Latent, or silent, reasoning lets language models carry out intermediate computation in continuous vector space instead of words, and is widely assumed to function as an internal scratchpad the model actively consults during inference.

By Ishan S. Kshirsagar

llms reinforcement-learning fine-tuning

arXiv Machine Learning

Jul 24

The Dark Room in the Reward Channel: Dense Prediction Rewards Collapse GRPO-Trained LLM Agents -- and What Actually Works

arXiv:2607. 21273v1 Announce Type: new Abstract: Dense per-step supervision is an appealing remedy for sparse-reward, long-horizon LLM agents: reward the agent for predicting its next observation, and memory should follow.

By Yu Wang

llms agents reinforcement-learning benchmarks

arXiv Machine Learning

Jul 24

Domyn-Small: A European 10B Reasoning Language Model

arXiv:2607. 20448v1 Announce Type: cross Abstract: We introduce Domyn-Small, a 10-billion-parameter open-weight reasoning language model released under the MIT license.

By Simone Angarano, Francesco Bertolotti, Federico D'Ambrosio, Michele Resta, Alessandro Rognoni, Nicol\`o Ruggeri, Dario Salvati, Andrea Valenti, Alberto Veneri, Martin Cimmino

llms agents reinforcement-learning fine-tuning benchmarks

arXiv AI

Jul 24

Evaluating Risks in Weak-to-Strong Alignment: A Bias-Variance Perspective

arXiv:2604. 25077v2 Announce Type: replace Abstract: Weak-to-strong alignment offers a promising route to scalable supervision, but it can fail when a strong model becomes confidently wrong on examples that lie in the weak model's blind spots.

By Hamid Osooli, Kareema Batool, Rick Gentry, Tiasa Singha Roy, Ashwin Gupta, Anirudha Ramesh

reinforcement-learning fine-tuning safety

arXiv AI

Jul 24

AINTMA: Agentic AI Architecture for Autonomous Test Management with Generative Intelligence, Secure Cloud Communication and Adaptive Quality Analytics

arXiv:2607. 20452v1 Announce Type: new Abstract: Modern software quality assurance demands intelligent, autonomous systems capable of adaptive decision-making across distributed cloud environments.

By Vinil Pasupuleti, Shyalendar Reddy Allala, Siva Rama Krishna Varma Bayyavarapu, Shrey Tyagi, Srinivasateja Songa

llms agents reinforcement-learning

arXiv AI

Jul 24

Robust Critics: Defending LLMs Against Multi-Turn Attacks

arXiv:2607. 20472v1 Announce Type: new Abstract: When a user asks a language model something harmful, is it a genuine attack or a misunderstood but well-meaning question?

By Roman Belaire, Arunesh Sinha, Pradeep Varakantham

llms reinforcement-learning fine-tuning safety

arXiv AI

Jul 24

TeaRAG: A Token-Efficient Agentic Retrieval-Augmented Generation Framework

arXiv:2511. 05385v2 Announce Type: replace-cross Abstract: Retrieval-Augmented Generation (RAG) utilizes external knowledge to augment Large Language Models' (LLMs) reliability.

By Chao Zhang, Yuhao Wang, Derong Xu, Haoxin Zhang, Yuanjie Lyu, Yuhao Chen, Shuochen Liu, Tong Xu, Xiangyu Zhao, Yan Gao, Yao Hu, Enhong Chen

llms rag agents reinforcement-learning

Get the daily briefing