PGPO: Potential-Guided Policy Optimization for Multi-Turn Agentic Tasks proposes a new reinforcement learning approach that estimates empirical state potentials from anchor-state-group return statistics within each rollout group. By deriving action advantages from potential differences between adjacent states, PGPO enables cross‑trajectory credit propagation, providing finer‑grained step‑level credit assignment, especially within failed trajectories. Experiments on ALFWorld and WebShop demonstrate strong overall performance compared to recent group‑based RL methods, with negligible training overhead.
IDEA: training-free Input-Dependent stEEring via Activation cluster matching (IDEEA) is a method that steers large language models by injecting bias into selected activations at inference time, without requiring weight updates. Unlike existing training-free steering approaches that use a single, input-independent direction, IDEEA clusters positive and negative activation supports per attention head and solves an optimal-matching problem to create a set of cluster-conditional directions. At inference, IDEEA selects the direction that best matches the input’s activation, aligning the model toward a target concept while preserving the input’s original representation, and achieves a 9.9% average improvement in truth × info rate on TruthfulQA compared to the best input-independent baseline.
The paper introduces a two‑stage training framework that combines Supervised Fine‑Tuning (SFT) and Direct Preference Optimization (DPO) to improve multimodal disaster severity assessment. It creates two datasets—ReasoningSet for validated rationales and PreferenceSet for paired rationales—using a single Human‑in‑the‑Loop workflow. Experiments on InternVL‑3‑8B and LLaVA‑1.5‑7B show that SFT boosts classification accuracy and Macro‑F1, while DPO further enhances interpretability and alignment with human judgment.
By Yuanjun Zhang, Fuzel Ahamed Shaik, Suvojit Acharjee, Fahad Khalid, Mourad Oussalah
arXiv:2509.25050v2 Announce Type: replace
Abstract: Reinforcement Learning (RL) has emerged as a central paradigm for advancing Large Language Models (LLMs), where both pre-training and RL post-train...
By Shuchen Xue, Chongjian Ge, Shilong Zhang, Yichen Li, Zhi-Ming Ma
arXiv:2609.00435v1 Announce Type: cross
Abstract: Cardiovascular sensing systems must preserve clinically useful information despite signal degradation, wireless losses, energy constraints, and edge-...
By Navaneeth Krishnan Kamalakannan, Janakiraman Kamalakannan, Harinisri Velmurugan
arXiv:2609.01354v1 Announce Type: cross
Abstract: Reinforcement learning with verifiable rewards (RLVR) and standard benchmark evaluation both rely on an automatic verifier that turns a free text ans...
By Esther Xin
Iterative GRPO is a batch‑online policy iteration framework that enables multi‑turn reinforcement learning for conversational agents without requiring an interactive user simulator. It alternates between learning a turn‑level Q‑function from logged returns (policy evaluation) and applying single‑turn GRPO against this Q‑function (policy improvement), thereby scoring candidate responses by their expected downstream return. The method is validated on six multi‑turn negotiation environments, demonstrating its practicality for real‑world deployment patterns.
By Daniel R. Jiang, Ankur Samanta, Yukai Yang, Jalaj Bhandari, R\'emi Munos, Tyler Lu
arXiv:2609.00213v1 Announce Type: new
Abstract: Reinforcement learning fine-tuning of large language models increasingly adopts multiple reward dimensions, including verifiable rules, task-specific e...
By Yu Yuan, Yaoyou Fan, Lili Zhao, Guangting Zheng, Kai Zhang, Lu Pan, Ke Zeng, Qi Liu
The paper introduces WM‑RMoE, a World Model‑based Risk‑aware Mixture‑of‑Experts framework for autonomous overtaking. It uses a learned latent dynamics model to perform multi‑step rollouts, evaluating cumulative risk at the trajectory level, and employs a hierarchical gating mechanism to coordinate long‑, short‑horizon, and rule‑based safety experts. A Gaussian Mixture Model preserves multimodal maneuver branches, improving robustness and preventing behavioral averaging, leading to better safety compliance, decision stability, and generalization in experiments.
By Yongzhi Liu, Sunan Zhang, Jinchang Xu, Jiawei Wang, Yushu Qiu, Chen Lv, Weichao Zhuang
arXiv:2609.00504v1 Announce Type: cross
Abstract: In this work, we study radically uncoupled learning in discounted general-sum Markov games. Assuming ``$\mathsf{ETH}$ for $\mathsf{PPAD}$", we show t...
By Asrin Efe Yorulmaz, Ugur Aydin, Tamer Basar
Neurosymbolics for Data Engineering introduces a neurosymbolic layer that can be added to existing LLM backbones to improve logical reasoning and reduce long‑context token usage. The layer boosts accuracy by an average of 85% on benchmarks such as BIRD‑CRITIC and LiveSQLBench without any task‑specific finetuning or RLHF. It also cuts effective token usage by over 50% and lowers time complexity from O(n²) to roughly O(n) for long‑context tasks.
By Vishvesh Bhat
PEARL is a framework that trains Socratic tutoring agents using pedagogically aligned reinforcement learning. It introduces a controllable student simulator to model diverse cognitive states, a reward model that jointly evaluates pedagogical quality and correctness, and a stable multi‑objective RL approach to balance competing tutoring goals. Experiments demonstrate that PEARL competes with both open‑source tutoring systems and leading proprietary LLMs.
By Qikai Chang, Zhenrong Zhang, Linbo Chen, Pengfei Hu, Jianshu Zhang, Youhui Guo, Jun Du
The paper introduces patterning, a reweighting technique that adjusts preference pairs based on their susceptibility to bias, to debias a Gemma 2 9B Instruct reward model trained on Skywork-Reward-Preference v0.2. Using this method, the authors achieve a +14.2 ± 1.2 percentage point improvement on the RM‑Bench Hard split while maintaining overall accuracy, matching the best reported Hard‑split gain from a comparable model. The study also demonstrates that the learned weights are interpretable, transferable across Gemma variants, and partially effective on Llama 3.1 8B.
By George Wang, Elizabeth Donoway, Daniel Murfet
arXiv:2510.14980v3 Announce Type: replace
Abstract: Large language models (LLMs) have shown strong abilities in writing and revising programs, yet many program-synthesis benchmarks still evaluate pro...
By Wenqian Zhang, Yangyi Huang, Weiyang Liu, Zhen Liu
The study investigates whether post‑training methods—GRPO, SFT, and DPO—improve language models’ ability to follow prompt evidence that conflicts with memorized knowledge. By comparing nine training variants across different scales and families, the authors find that grounding gains are modest for GRPO, moderate for Conflict‑SFT, and near‑ceiling for DPO, but all largely rely on the same causal attention‑head set present in the starting checkpoint. Removing the starting‑model grounding direction suppresses these gains, while adding it back recovers a significant portion of DPO’s improvement, indicating that existing model machinery drives most of the observed gains.
By Prakhar Gupta, Vaibhav Gupta
The article argues that conversational AI should provide contingent feedback—responses that vary with user behavior and its social consequences—rather than merely seeking user approval and fluency. It highlights how current alignment methods, such as reinforcement learning from human feedback, often produce sycophantic, noncontingent affirmation, which can hinder the development of interpersonal skills, especially in adolescents. The authors propose a framework for evaluating and designing contingent AI, incorporating trajectory-based assessment and social consequence prediction, and call for interdisciplinary research to ensure AI systems positively influence human social learning.
By Scott Compton, Arjun Nagendran
arXiv:2609.00189v1 Announce Type: new
Abstract: Goal-directed optimization is essential for steering molecular generators to propose candidates with desired properties. However, it is often implement...
By Shiyun Wa, Yifei Wang, Anna G. Green, Simone Sciabola, Ye Wang
arXiv:2606.21811v2 Announce Type: replace-cross
Abstract: Coding tasks are typically complicated and require multiple capabilities, ranging from high-level planning to low-level implementation. While...
By Shubham Gandhi, Yiqing Xie, Atharva Naik, Ruichen Zhu, Carolyn Rose
The paper introduces a stage‑wise closed‑loop evaluation method to track how compression techniques affect learned driving policies. Using a POMDP formulation and PPO‑trained belief‑state policy in Gym‑Duckietown, the authors compress the actor step by step and test it across five driving curricula. They find that structured pruning first degrades driving capability, distillation can partially recover it but is limited by rehearsal data, and integer quantization further loses curricula requiring stopping and resuming, whereas the unpruned actor retains all curricula.
By Ahmad Alfan Alfian Irfan, Nur Ahmad Khatim, Mansur Arief
The paper introduces a discrete‑time Markov decision process (DTMDP) model for a finite‑horizon, multi‑item capacitated lot‑sizing problem where demand quantities are deterministic but demand‑arrival times are stochastic. It compares stochastic instances to deterministic counterparts, showing that stochastic timing significantly enlarges the state space, transitions, solution time, and memory usage. A genetic algorithm (GA) is proposed to search feasible state‑feedback policies, achieving an average optimality gap of about 3.44 % and a speedup of roughly 6.89× on challenging benchmark instances.
By L\'ea Bayati, Mohamed Dahmoune, Melek Rodoplu