The paper extends the concept of social laws from deterministic, goal-based multi‑agent systems to stochastic, reward‑based environments. It introduces a formalism for defining and verifying the robustness of these laws, including a new metric called α‑robustness that quantifies the utility each agent can guarantee while following the law. The authors present a verification approach that reduces the problem to solving multiple Markov decision processes and demonstrate the framework’s potential through empirical evaluations on toy environments.
By Rolando Fernandez, Caleb Probine, Tyler Lee, Jeffrey Chen, Erez Karpas, Muhammad Arrasy Rahman, Peter Stone, Ufuk Topcu
The paper identifies a failure mode in Proximal Policy Optimization (PPO) critics called Value Flattening, where state values vary sharply across intermediate states while critic predictions remain flat. The authors analyze this phenomenon theoretically and empirically, linking it to an implicit variance penalty and redundant updates from temporally correlated states. They propose Sparse Proximal Policy Optimization (SP³O), which applies the value loss to only a few well‑separated states, and demonstrate that this sparse supervision mitigates Value Flattening and improves policy performance on Qwen3‑Base across model sizes and evaluation suites.
By Yizhuo Li, Jianhao Yan, Yun Luo, Zhi Wang, Futing Wang, Rong-Xi Tan, Kanghui Tian, Ganqu Cui, Ning Ding, Peilin Zhao, Yafu Li, Yu Cheng
CERA-MoA introduces a co-evolving router and independently learning agents for Mixture-of-Agents systems, allowing routing strategies to adapt to evolving agent capabilities. The framework uses a predictive familiarity estimator based on mid-layer hidden states to evaluate semantic competence, and a cumulative-threshold adaptive routing mechanism to activate a minimal agent subset. Experiments across multiple domains show that CERA-MoA outperforms static-agent routing and fixed-workflow fine-tuning baselines.
By Jiaxuan Jiang, Liyuan He, Zhixuan Fang
REVERSAL-BENCH is a benchmark that introduces a continuous reversibility parameter ρ∈[0,1] and a reset oracle to evaluate how well reinforcement learning agents can recover from irreversible states across eight manipulation tasks in five physics engines. Experiments show a sharp reversibility cliff: reset‑free agents become trapped in irrecoverable states as ρ increases, while episodic agents continue learning steadily. The benchmark also provides a large multi‑simulator dataset and demonstrates that safety shields can predict recoverability but only succeed when the agent can avoid the trap.
By Riyaaz Shaik, Chandru Venkataraman
The paper presents a reinforcement‑learning framework that dynamically selects between full‑order models (FOMs) and reduced‑order models (ROMs) in hybrid domain decomposition simulations using the overlapping Schwarz alternating method (O‑SAM). Offline‑trained Deep Q‑networks choose subdomain‑local FOMs or pre‑trained Operator Inference ROMs based on a reward that balances accuracy, computational cost, and model‑switching frequency, and the learned policies are deployed on unseen problem instances without needing a reference FOM solution. Experiments on a 1D advection‑diffusion problem and a 3D elastic wave propagation benchmark show that the RL‑guided policies adaptively allocate high‑fidelity resolution as features propagate, outperforming static FOM/ROM assignments and demonstrating the feasibility of predictive online fidelity adaptation in Schwarz‑based hybrid simulations.
By Trishit Mondal, Irina Tezaur, Anthony Gruber
The paper introduces Mask 2D-3D, an Adaptive Dual-Masked Autoencoder Network designed for image-to-point cloud registration. It proposes an Intermodal Dual-MAE Framework (ID-MAE) with a Similarity-based RL Masking Strategy (SRLM) that adaptively masks informative positions using cross-modal similarity and reinforcement learning. Experiments on RGB-D Scenes v2 and 7-Scenes benchmarks demonstrate state-of-the-art performance in this registration task.
By Zhixin Cheng, Jiacheng Deng, Xiaotian Yin, Baoqun Yin, Richang Hong, Tianzhu Zhang
The paper introduces APGEM, an adaptive controller that dynamically selects among four error‑mitigation techniques—Zero‑Noise Extrapolation, Probabilistic Error Cancellation, Clifford Data Regression, and Readout Error Mitigation—based on a utility function and Q‑learning scores. Applied to a realistic Delhi‑based Capacitated Vehicle Routing Problem, the adaptive approach improves the quantum reinforcement learning agent’s approximation ratios from 0.84‑0.87 to 0.92‑0.94 under high noise, outperforming constructive heuristics and approaching metaheuristics. The controller’s strategy shifts from a Clifford‑data‑regression‑heavy regime early in training to a balanced use of all techniques as training progresses, demonstrating regime‑dependent selection.
By Shabir Ahmad Sofi, Bisma Majid, Mir Mohammad Yousuf
Changepoint-Aware World Models (CAWM) is a DreamerV3 agent that detects abrupt dynamics shifts in a robot’s environment using an online CUSUM test on internal prediction error. Upon detection, CAWM selectively forgets stale replay data while preserving the learned representation, enabling rapid recovery from shifts such as doubled gravity or halved actuator gain. Experiments on simulated locomotion show CAWM recovers faster than passive retraining and outperforms a baseline that respawns a fresh dynamics model, achieving significant return gains in the first 30k post‑shift frames.
By Everest Yang
EfficientTDMPC is a sample‑efficient, model‑based reinforcement learning method for continuous control that builds on the TD‑MPC family. It reduces estimation error by using an ensemble of dynamics models and averaging return estimates across models and rollout depths, and it can penalize uncertainty in the planner objective. Practical improvements such as fresher buffer data and reduced compute enable the algorithm to benefit from a higher update‑to‑data ratio, achieving state‑of‑the‑art sample efficiency on HumanoidBench‑Hard and DMC hard while matching state‑of‑the‑art on DMC easy.
By Thomas Evers, Cristian Meo, Wendelin Bohmer, Justin Dauwels, Yaniv Oren
The paper introduces a reinforcement‑learning based rewriting agent that rewrites training data to reduce the distribution mismatch between supervised fine‑tuning (SFT) and a language model’s generation distribution. By formulating data rewriting as a policy‑learning problem, the authors train a lightweight LoRA rewriting policy that optimizes alignment with question‑answering style, maintains semantic diversity, and enforces task consistency. Experiments on three instruction‑tuned backbones show that models fine‑tuned with the rewritten data achieve downstream performance comparable to standard SFT while mitigating degradation on non‑downstream benchmarks, and preliminary tests suggest the policy can transfer across domains such as logical reasoning and medical question answering.
By Jiacheng Wang, Zhijie Liu, Ping Jian, Zirong Chen, Ke Ren Liao, Zhen Yang, Zhongbin Guo
The paper introduces a multitask large reasoning model for molecular science that incorporates chemical knowledge via a multispecialist architecture, chain-of-thought supervision, and molecule-informed reinforcement learning. It coordinates prediction and inference specialists across ten molecular tasks—including description, generation, nomenclature translation, property prediction, and reaction prediction—using task-conditioned routing. The model surpasses more than 20 general-purpose and molecular large language models, improving aggregate performance by 50.3% and outperforming leading multitask baselines on most tasks, while maintaining interpretable chemical inference and demonstrating a workflow for CNS candidate generation and retrosynthetic planning.
By Pengfei Liu, Shuang Ge, Xiaobo Wang, Xin Liu, Jun Tao, Yan Li, Chao Liu, Ling Chen, Zhixiang Ren
arXiv:2407.06868v3 Announce Type: replace-cross
Abstract: In this work, we propose a method for efficient resource utilization of simultaneously transmitting and reflecting reconfigurable intelligent...
By Ashok S. Kumar, Nancy Nayak, Sheetal Kalyani, Himal A. Suraweera, Lajos Hanzo
The paper introduces SARA, an industrial framework that scales articulated user rationales (AURs) for recommendation systems. It curates a high‑quality AUR dataset from 240 M users, trains a 7B‑parameter MLLM (SARA‑7B) to generate rationales for millions of authors, and integrates these generated rationales into a production ranking model (SARA‑Ranker). Offline and online experiments demonstrate that the system produces more specific, polarity‑consistent rationales and improves user engagement while reducing negative feedback.
By Haoke Xiao, Yueyang Liu, Yuhui Zhang, Xiang Chen, Yufei Liu, Jia Xu, Yalong Guan, Xiaolan Zhu, Xiaoyu Zhang, Shijun Wang, Shuang Yang, Zijie Meng, Zejian Zhang, Ruochen Yang, Xiangyu Wu, Tingting Gao, Han Li, Lantao Hu, Cheng Luo, Kun Gai
The paper addresses the memory bottleneck in reinforcement learning for large language models caused by the large Key-Value (KV) cache during rollout phases. It highlights that while KV cache compression can reduce memory usage, it introduces a significant off‑policy bias that standard statistical corrections cannot adequately mitigate. The authors argue that even tiny compression errors are amplified by RL’s instability, leading to inefficient learning.
By Rui Zhu, Weiheng Bai, Qiushi Wu, Yang Ren, Haixu Tang, Yuchu Liu
The paper introduces label‑free bias‑only test‑time reinforcement learning (TTRL), which uses majority‑vote pseudo‑labels as rewards and optimizes only about 100 K bias parameters while keeping the pretrained backbone frozen. On the MATH‑500 benchmark it achieves 76.67 % accuracy, slightly better than a labeled bias‑steering baseline, and improves performance on several vision‑language and audio reasoning tasks. The authors also show that the learned steering vectors transfer to 4,500 held‑out MATH problems and analyze why such a highly restricted adaptation works, linking majority‑vote reliability to rollout consensus and gradient energy in bias subspaces.
By Naveen Vakada, Mingyuan Li, Shaoxiong Ji
FedGuide is a federated reinforcement learning framework that tackles distributional mismatch among heterogeneous clients by using diffusion priors as personalized behavior models. It aggregates these priors via an Optimal-Transport Mixture-of-Experts, preserving diverse behavior modes, and introduces a Distribution Correction Estimation (DICE) value baseline to give low‑variance, return‑aware guidance for local policy improvement. Experiments on heterogeneous environments demonstrate that FedGuide surpasses existing FRL methods in client‑average returns, final‑round performance, and worst‑round robustness while maintaining stable learning under stronger heterogeneity.
By Zhilin He, Gauri Joshi
Safety tuning pipelines judge only the final answer, which makes it difficult to distinguish robust refusal from two undesirable shortcuts: blanket refusal on benign requests and polished but unfaithf...
Reinforcement learning (RL) is an exciting concept as well as a remarkable success story worth sharing. However, RL builds on rather complex interactions between different objects that play out over s...
Current Mixture-of-Agents (MoA) paradigms generally treat query routing and agent fine-tuning as separate processes, limiting their ability to respond to evolving agent capabilities. This disconnect p...
Emergency management assistance programs, such as relief distribution, are essential for delivering necessary supplies to affected communities. However, these programs operate in a decentralized netwo...