Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

6,937 stories · RSS feed

arXiv AI
Aug 25

HiDiffTIR: Hierarchical Difficulty-Aware Policy Optimization for Multi-Turn Tool-Integrated Reasoning

arXiv:2608.21863v1 Announce Type: cross Abstract: Tool-Integrated Reasoning (TIR) is a fundamental capability for LLM agents to solve complex tasks by interacting with external tools iteratively. Rei...

By Yucan Guo, Xiaohan Wang, Miao Su, Saiping Guan, Zhongni Hou, Jiajun Chai, Wei Lin, Guojun Yin, Xiaolong Jin, Jiafeng Guo, Xueqi Cheng
arXiv AI
Aug 25

GeoRisk-RAG: A Hierarchy-Aware Risk Framework for Improving RAG Reliability through Selective Answering

GeoRisk-RAG is a hierarchy‑aware framework that improves the reliability of Retrieval‑Augmented Generation (RAG) by incorporating geographic validity. It estimates geographic applicability using a Directed Acyclic Graph (DAG)‑based distance during context retrieval, enabling selective answering. Experiments on a wildfire‑related QA dataset show that GeoRisk‑RAG reduces false confidence rates from ~0.090 to 0.009 and aligns better with human preferences.

By Meenu Ravi, Shailik Sarkar, Lulwah AlKulaib, Yordanos Tessema, Chang-Tien Lu
arXiv AI
Aug 25

AdaR: A Framework for Equipping LLMs with Adaptive Reasoning

AdaR is a framework designed to enhance large language models (LLMs) with adaptive reasoning for mathematical tasks. It identifies and mitigates spurious reasoning—where models rely on superficial correlations—by generating logically equivalent queries and training with Reinforcement Learning with Verifiable Rewards (RLVR) to penalize incorrect logic and promote adaptive logic. The approach includes extracting problem‑solving logic, executing code to verify answers, and applying sanity checks, resulting in significant gains in mathematical reasoning performance and improved data efficiency.

By Zhejian Lai, Xiang Geng, Zhijun Wang, Yang Bai, Jiahuan Li, Rongxiang Weng, Jingang Wang, Xuezhi Cao, Xunliang Cai, Shujian Huang
arXiv AI
Aug 25

Is Next-Chunk Reasoning RL Really Better than SFT? Revisiting Training Strategies under no-CoT Data

arXiv:2608.23256v1 Announce Type: new Abstract: Recent work proposes next-chunk reasoning RL for leveraging no-CoT data---corpora such as worked solutions and textbook derivations that contain reason...

By Yinhao Tang, Youqing Fang, Yanan Sun, Jiangning Liu, Ziyi Wang, Xun Zhao, Weiming Zhang, Bin Liu, Kuikun Liu, Wenwei Zhang, Kai Chen
arXiv AI
Aug 25

SRMT: Shared Memory for Multi-agent Lifelong Pathfinding

The paper introduces the Shared Recurrent Memory Transformer (SRMT), a decentralized multi‑agent reinforcement learning framework that uses a global memory workspace for agents to broadcast and query each other’s learned states. SRMT is evaluated on the Partially Observable Multi‑Agent Pathfinding (PO‑MAPF) problem, showing that shared memory enables emergent coordination even with minimal reward guidance and outperforms existing baselines on the Bottleneck task and scales competitively on larger POGEMA maps. The authors provide open‑source code for training and evaluation on GitHub.

By Alsu Sagirova, Yuri Kuratov, Mikhail Burtsev
arXiv AI
Aug 25

Jagged Judges: Epistemic Stability Under Perturbation, Pressure, and Persistence

The paper introduces the Wiggle Framework, a unified stress test for assessing epistemic stability in large language model (LLM) judges. It evaluates judge robustness across three dimensions—Mechanical Consistency, Single-turn Conviction, and Multi-turn Persistence—using 9 frontier models on 14 judging tasks related to safety, toxicity, AI writing detection, and political-response evaluation. Results show significant instability, with verdict flips ranging from 25–71% under static pushback and 62–91% when challenged by an adversarial LLM, and highlight that successful pressure often misaligns with ground truth.

By Justin Zhao, Himaghna Bhattacharjee, Hannah Korevaar, Bhaktipriya Radharapu, Khalid El-Arini