The paper introduces a foundation model–aided actor‑critic algorithm to boost multi‑agent reinforcement learning (MARL) for optimizing random access (RA) wireless networks. By embedding a foundation model into a consensus‑based decentralized MARL framework, the authors provide convergence analysis that matches conventional MARL under local reward exchanges and nonlinear value function approximations. Numerical experiments demonstrate that this FM‑based approach markedly accelerates MARL training for RA network optimization.
By Myeung Suk Oh, Zhiyao Zhang, Alvaro Velasquez, Nathaniel D. Bastian, Jia Liu
The paper introduces Robust Async-Fed-Q, an epoch‑based federated Q‑learning algorithm that combines variance‑reduced Bellman operator estimation with robust server aggregation. It provides high‑probability finite‑time guarantees showing that honest agents retain sample‑efficiency benefits even when some agents act adversarially, with the adversarial impact diminishing as honest data grows. The authors also present matching lower bounds, extend the framework to single‑trajectory sampling and heterogeneous coverage, and improve communication complexity for asynchronous federated Q‑learning.
By Sreejeet Maity, Aritra Mitra
The paper identifies that low‑timestep deep spiking Q‑networks (DSQNs) suffer mainly from common‑mode errors that degrade Q‑value estimation. To address this, the authors introduce Common‑Mode Compensation DSQN (CMC‑DSQN), which employs an auxiliary ANN during training to correct these errors while keeping inference energy‑efficient. Experiments on Atari and MiniAtar show that CMC‑DSQN improves performance by nearly 20 % at two timesteps and outperforms both DSQN baselines and the ANN baseline at four timesteps.
By Zijie Xu, Bingrui Guo, Yiding Sun, Yiting Dong, Zhile Yang, Zhaofei Yu
QF3 (Fast Flow RL with Filtered Q-Gradients) is an online off‑policy reinforcement learning algorithm that trains flow policies by combining flow matching with the critic’s action gradient, backpropagated through a one‑step prediction of the flow output. It selectively applies the critic gradient only to action dimensions close to the replay action to ensure reliable updates. QF3 is the first off‑policy flow RL method to train humanoid locomotion policies from scratch and transfer them zero‑shot to hardware, achieving a 10× wall‑clock speedup over the on‑policy FPO++ method, and it can also fine‑tune pretrained flow‑based manipulation policies on ABC‑Sim and Robomimic tasks.
By Chung Min Kim, Brent Yi, David McAllister, Hongsuk Choi, Himanshu Gaurav Singh, Jinkun Cao, Ken Goldberg, Pieter Abbeel, Carmelo Sferrazza, Angjoo Kanazawa
The paper introduces PAIR, a pipeline that creates user-conditioned application states to evaluate mobile GUI agents across different users, and RePAIR, a reinforcement learning method that incorporates personalization-aware interaction rewards. Experiments with six agents reveal significant variability in task success and subgoal achievement across users, especially when targeting personal content. RePAIR improves success rates over supervised fine-tuning, suggesting that learning from cross-user differences can enhance GUI-agent reliability.
By Yeji Park, Jaeyun Shim, Taesik Gong
SIGMA is a self‑improving alignment framework that uses a model’s own reasoning to generate diverse alignment dilemma scenarios from a simple "Model Spec" and then fine‑tunes the model with self‑judged rewards. By performing spec‑guided task synthesis and rubric‑based reinforcement learning, SIGMA improves safety alignment in multi‑turn agentic settings, reducing harmfulness and misalignment scores while preserving general capability. The approach demonstrates that a model can autonomously strengthen its own safety reasoning without external supervision.
By Jingyu Zhang, Shruti Palaskar, Daniel Khashabi, Benjamin Van Durme, Leon A. Gatys, Joseph Yitan Cheng
The paper introduces SUTURE, a structured verification method for video temporal grounding that leverages the joint structure of rollout groups rather than scoring each rollout independently. SUTURE conditions verification on the entire rollout group, using disagreement across rollouts to reweight targets and coverage at each position to redistribute reward mass. Experiments on five benchmarks show that SUTURE consistently improves grounding performance across all IoU thresholds and reduces video-start anchoring in reasoning traces, indicating that the joint structure of a rollout group can provide a more informative temporal verifier.
By Youngjae Cho, Won Young Jhoo, Jongsuk Kim
BluffJAX is an open‑source suite of adversarial imperfect‑information games implemented in JAX, designed for high‑throughput simulation and GPU parallelization. It includes classic benchmarks like Texas Hold’Em and Kuhn Poker, as well as newer games such as Bluff, Stud Poker, and Kemps, aiming to broaden research in game‑theoretic reinforcement learning. The authors benchmarked throughput, memory usage, and algorithmic performance across single and multi‑GPU setups, showing scalability to hundreds of millions of samples per second and providing baseline results for future work.
By Aryaman Reddi, Jan Peters, Carlo D'Eramo
The paper presents a continuous Dyna loop that trains a multi‑agent world model for a ten‑hero MOBA game and evaluates the resulting policy in the real game. By using imagined episodes and real‑game data to continuously refine the model, the policy achieves a 70.2% win rate as radiant on unseen seeds, a dramatic improvement over dream‑only training. The study highlights that internal dream metrics can miss model exploitation, that the world model must be repaired on the policy’s own games, and that the policy’s success stems from macro‑level pressure rather than precise micro‑tactics.
By Jordy Kieto
The paper introduces SSRFT (Supervised Safe-Role Fine‑Tuning), a new framework that reframes safety alignment for large language models as internalizing a predefined safe role rather than relying on explicit refusal patterns. SSRFT builds a Safe‑Role Question‑Answer dataset from psychometric questions, limited jailbreak prompts, and a safe‑role description, then synthesizes and validates role‑consistent responses to expose models to diverse safety scenarios. Experiments on multiple base and instruct models demonstrate that SSRFT yields more robust and generalizable safety alignment, better resilience to prefilling attacks, improved generalization to unseen jailbreak domains, and reduced over‑refusal on benign queries while preserving general capabilities.
By Jinghao Pang, Jitai Hao, Qiang Huang, Zhaochun Ren, Jun Yu
arXiv:2610.07043v1 Announce Type: cross
Abstract: Low-precision execution can substantially accelerate reinforcement learning (RL) for large language models, but discrepancies between learner and sam...
By Zhen Li, Shuai Zhang, Yanggan Gu, Yiming Zhang, Yang Yu, Mingfa Feng, Congkai Xie, Shuang Yu, Junjie Lai, Hongxia Yang
arXiv:2610.07286v1 Announce Type: cross
Abstract: Model optimizations help improve inference performance and accuracy of ML workflows. However, relying on a single model to perform inference across a...
By Abhilash Jindal, Todd Nief, Bhanu Prakash Vangala, Shankaradithyaa V, Tvisha Malik, Anshik Sahu, Aaron Schein, Amitabh Chaudhary, Tanu Malik
arXiv:2610.07899v1 Announce Type: cross
Abstract: Generative actors are transforming offline reinforcement learning (RL) by enabling expressive policy classes that model complex action distributions....
By Guhyeon Kang, Minhae Kwon
arXiv:2610.08781v1 Announce Type: cross
Abstract: Scientific research often begins by synthesizing ideas from a set of related papers to identify gaps and formulate new directions. However, training...
By Ziyu Chen, Yilun Zhao, Jiashuo Sun, Yiling Ma, Manasi Patwardhan, Arman Cohan
arXiv:2511.21075v4 Announce Type: replace-cross
Abstract: Engineering LLMs to accelerate life sciences research requires a robust alignment with biomedical knowledge. We observe that biomedical text...
By Zhenchao Tang, Fang Wang, Haohuai He, Jiale Zhou, Tianxu Lv, Jun Zhu, Shouzhi Chen, Minghao Yang, Yu Wang, Jiayang Wu, Yidong Song, Yaokun Li, Jiehui Huang, Jun Zhou, Bing He, Jianhua Yao
arXiv:2610.01936v2 Announce Type: replace
Abstract: Large Language Models (LLMs) have demonstrated remarkable fluency and versatility across natural language tasks but remain fundamentally limited by...
By Meghana Sunil, Shravya V, Shravan Venkatraman, Joe Dhanith PR
arXiv:2602.16863v3 Announce Type: replace-cross
Abstract: The ability to manipulate tools significantly expands the set of tasks a robot can perform. Yet, tool manipulation represents a challenging c...
By Kushal Kedia, Tyler Ga Wei Lum, Jeannette Bohg, C. Karen Liu
arXiv:2603.04364v2 Announce Type: replace-cross
Abstract: Multimodal web agents that process both screenshots and accessibility trees are increasingly deployed to interact with web interfaces, yet th...
By Haoyu Liu, Dingcheng Li, Lukas Rutishauser, Zeyu Zheng
arXiv:2605.07079v2 Announce Type: replace-cross
Abstract: World models predict future transitions from observations and actions. Existing works predominantly focus on image generation only. Visual fe...
By Xinyu Zhang, Zhengtong Xu, Yutian Tao, Yeping Wang, Yu She, Abdeslam Boularias
arXiv:2610.06931v1 Announce Type: new
Abstract: In this paper, we study the sample complexities of value and policy learning in finite discounted Markov decision processes (MDPs) under recursive entr...
By Amirparsa Bahrami, Oliver Mortensen, Mohammad Sadegh Talebi