The paper introduces TriTrust-PBRL (TTP), a framework for preference-based reinforcement learning that learns a shared reward model and expert-specific trust parameters from multi-expert feedback. Trust parameters evolve to be positive (trust), near zero (ignore), or negative (flip), allowing the model to invert adversarial preferences instead of discarding them. Experiments on manipulation and locomotion tasks show TTP achieves near-oracle performance under adversarial corruption, outperforming existing PBRL methods.
By Seyed Amir Hosseini, Maryam Abdolali, Amirhosein Tavakkoli, Fardin Ayar, Ehsan Javanmardi, Manabu Tsukada, Mahdi Javanmardi
The paper presents a reinforcement learning framework for strategic investment decisions in the energy transition, aiming to balance profit, CO2 social costs, and competitive advantage across oil & gas, renewables, and CO2 reduction sectors. A custom simulation environment models the dynamic energy landscape up to 2050, allowing the RL agent to learn optimal policies that adapt to uncertainties in production, prices, and costs. Benchmarking shows the RL strategy consistently outperforms manually defined baseline policies in adaptability and long-term value creation.
By Yasaman Cheraghi (Department of Energy and Petroleum Engineering, University of Stavanger, Norway), Reidar B. Bratvold (Department of Energy and Petroleum Engineering, University of Stavanger, Norway), Aojie Hong (Independent Researcher, Stavanger, Norway), Ressi B. Muhammad (Department of Energy and Petroleum Engineering, University of Stavanger, Norway), Sergey Alyaev (NORCE Norwegian Research Centre, Bergen, Norway)
SAIL is an open model with 35 B total and 3 B active parameters designed for literature research, scientific coding, and multi‑step research workflows. It is developed through a science‑aware improvement loop in which agents built on frontier AI models analyze task failures, create training tasks to address capability gaps, and iterate this process across multiple development cycles. The training pipeline combines supervised fine‑tuning, specialist training, multi‑teacher on‑policy distillation, and agentic reinforcement learning, enabling SAIL to achieve competitive performance on scientific research tasks with fewer parameters than leading open‑weight models.
By SAIL Model Team, Boyuan Sun, Bryan Dai, Che Liu, Chi Liu, Derek Li, Hongming Piao, Mengzhuo Chen, Xidong Wang, Yan Shu, Yinda Chen, Ziyang Zeng
AgentGarten is a framework that links simulators and game engines with a shared neural renderer to create real‑time interactive virtual worlds. It uses a pretrained video model adapted to geometry conditions, refined with Adversarial Forcing, to produce realistic visual observations while maintaining consistent state and rules. The system enables agents to learn efficiently—achieving comparable performance in only four training rounds versus millions in traditional reinforcement learning—by distilling experience into playbooks that successive agents can inherit and improve upon.
By Jiawei Chi, Shangchen Miao, Zhiyuan Shi, Kailu Wu, Hanyang Wang, Weiliang Chen, Qiyu Dai, Jinshan Ren, Jun Gao, Mingsheng Long, Yueqi Duan, Jiangran Lyu, Jialong Wu, Fangfu Liu
MiMo‑V2.6 is an omni‑modal reinforcement learning framework that scales compute to advance large foundation models toward self‑improvement. It trains on a broad multimodal corpus before RL, then scales RL along three axes: larger batches and throughput, diverse environments across code, general, visual, and cyber domains, and enhanced grader compute for better reward signals. The system incorporates infrastructure such as a frozen MoE router, multi‑layer defense against reward hacking, unified trajectory representation, high‑concurrency rollout, and training‑inference consistency, and it is open‑source to support reproducibility and further research.
By Core Team, Zongming Qiao, Ziyue Hua, Zirui Ou, Zihao Yue, Zihan Jiang, Zhuo Huang, Zhiyang Chen, Zhixian Zheng, Zhipeng Xu, Zhengrui Ma, Yuyang Hu, Yuhang Dong, Yuechen Zhang, Yudong Wang, Yuanxin Liu, Yixin Yang, Yishuo Cai, Yikai Zhao, Yihan Yan, Yifan Zhang, Yifan Song, Xiyu Wei, Xing Zhang, Xin Zhang, Xiaoqian Liu, Xiaodong Ji, Xiangwei Deng, Xueyu Guo, Wenhan Ma, Weimin Xiong, Weikun Wang, Weiji Zhuang, Shuo Liu, Shuhuai Ren, Shuhao Gu, Shimao Chen, Shijie Cao, Shihua Yu, Shicheng Li, Shengjie Zhou, Shaolei Zhang, Rang Li, Qiying Wang, Qingkai Fang, Qianli Chen, Minzheng Wang, Liwen Wang, Linli Yao, Linghao Zhang, Liangyu Cheng, Liang Zhao, Lei Li, Jinhao Dong, Jinyu Xiang, Jianyu Wei, Jiangshan Duo, Huaqiu Liu, Huanjie Fan, Hongyi Guan, Hongshen Xu, Hao Tian, Hanyu Li, Hailin Zhang, Gang Wang, Fuli Luo, Feng Wei, Dong Zhang, Dawei Zhu, Chiheng Lou, Chenhong He, Chenhao He, Chenghua Liu, Bowen Ye, Bowen Shen, Boshen Xu, Bo Yang, Bingquan Xia, Bangjun Xiao, Baixuan Xu, Zhouxiang Mao, Zhiyang Zhang, Zhixiang Xu, Zhenru Lin, Zhengju Tang, Zhaojun Huang, Yuzhe Weng, Yuxing Xiang, Yuxiao Li, Yuheng Yang, Yuhang Wang, Yuchen Liu, Yuanyuan Tian, Yuanliang Dong, Yu Cheng, Yongzhe He, Yongshun Liang, Yong Wang, Yiyan Wang, Yitian Gong, Yijie Zhang, Yanshu Xin, Xun Zhang, Xingjian Zhao, Wenyu Yang, Wenshan Huang, Wenhao Li, Tingwei Huang, Tianyu Yu, Tianyang Lu, Taoyu Yang, Sinan Du, Shutong Tian, Shulin Du, Shengfan Wang, Shanchuan Fang, Qihao Zhang, Qibin Yang, Qian Yu, Qian Tu, Pengrong Xie, Peipei Wang, Peidian Li, Minkun Guo, Mingchen Shao, Luohan Gao, Lijie Wang, Liang Shi, Kaiqi Chen, Kaiming Liu, Kaifei Wang, Kai Yang, Jinlong Xue, Jiechen Zhang, Jiaxuan Liu, Hongxu An, Hao Peng, Hanglong L\"u, Guonan Wang, Feiyu Yang, Fanyu Cao, Fangyue Liu, Fan Cui, Cong Wang, Chun Chen, Chenxu Bai, Chengxuan Zhu, Chenghua Wang, Boyi Zeng
VICO proposes a co‑evolutionary framework where a vision‑language model (actor) and an Environment‑as‑Rewriter (EnvRewriter) are trained together. The EnvRewriter edits visual structures—such as scene graphs, chart tables, or protected region masks—to generate new training samples whose difficulty is calibrated to the actor’s current performance via a pass‑rate reward. Across nine multimodal benchmarks, VICO‑8B outperforms its base model by up to +5.0 % on out‑of‑domain tasks and surpasses other co‑evolution baselines while requiring far fewer labeled samples.
By Meng Lu, Ligeng Zhu, Olivia Xiao, Yuchen Zhuang, Zihan Wang, Kuncheng Wu, Bangya Liu, Yu Wang, Charles Fleming, Wenqi Shi, Xuan Wang
The paper introduces Proof-of-Use (PoU), a reinforcement learning framework designed to prevent tool‑call hacking in deep research agents. PoU enforces evidence grounding by requiring agents to cite normalized evidence identifiers and uses a multi‑objective reward system that includes progressive process rewards, an answer‑support alignment reward, and a curriculum‑style adaptive reward mixing. Experiments demonstrate that PoU effectively mitigates tool‑call hacking and promotes robust, adaptive tool usage even under domain and tool shifts.
By Shengjie Ma, Chenlong Deng, Jiaxin Mao, Jiadeng Huang, Teng Wang, Junjie Wu, Changwang Zhang, Jun wang
The paper introduces the Viability‑constrained Behavioral Steering Attack (V‑BSA), a new attack strategy for federated reinforcement learning that targets the timing of interventions rather than just the construction of malicious updates. By selecting sparse intervention states based on local policy uncertainty and applying envelope‑constrained steering, V‑BSA can significantly degrade performance on discrete‑action benchmarks while using far fewer interventions than traditional dense poisoning attacks. The study demonstrates that the specific states at which interventions occur constitute a distinct and critical dimension of robustness in sequential decision‑making settings.
By Shutong Zheng, Sijia Chen
The study investigates how spatial patterns arise when agents in public goods dilemmas learn movement strategies via tabular Q‑learning. Cooperators tend to cluster around resource peaks, while co‑adaptation with defectors can alter cluster strength and motion, sometimes producing traveling bands. Higher cooperator learning rates combined with lower defector rates lead to the greatest welfare losses, and crowding penalties can mitigate these losses, linking learning rates to spatial organization and collective welfare.
By Yefei Zhang, Yuxuan Zhao
DVLA-RL++ introduces Dual-Level Vision‑Language Alignment with Reinforcement Learning Gating for few‑shot learning. It adds complementary semantic purification (CSP) to separate intrinsic from nuisance textual descriptions and counterfactual reinforcement‑learning gating (CRG) to learn layer‑wise fusion strengths. Experiments on standard, fine‑grained, and cross‑domain benchmarks demonstrate state‑of‑the‑art accuracy, improving on DVLA‑RL by an average of 1.4%.
By Wenhao Li, Xianjing Meng, Qiangchang Wang, Zhongyi Han, Yilong Yin, Liqiang Nie
The paper proposes a new loss scheme that simultaneously supervises zeroth- and first-order actions to improve policy performance and control robustness in data-driven decision-making methods such as imitation learning and reinforcement learning. The approach provides formal theoretical guarantees, works with any off-the-shelf policy model, and can be added as a lightweight plug‑and‑play module to existing offline RL frameworks. Experiments on OGBench and D4RL show significant gains in performance, robustness, and out‑of‑distribution generalization, especially in low‑data settings.
By Peng Cheng, Yunxian Hou, Zhi Zhou, Qian Zhang, Chang Huang, Xianyuan Zhan
The paper introduces Adaptive Reference Guidance (ARG), a method that balances using a verified reference solution with free generation in trajectory rollouts for reasoning reinforcement learning. ARG selects a prefix of the reference to guide the model, continuing from it only if the resulting trajectory passes verification, otherwise falling back to the reference. Experiments on Qwen3-4B and Qwen3-8B across five mathematical reasoning benchmarks demonstrate that ARG achieves the highest aggregate pass@12 while maintaining competitive average sampled accuracy.
By Hanyu Wang, Nakul Agarwal, Hossein Nourkhiz Mahjoub, Ehsan Moradi Pari, Makoto Fukushima, Jinghui Chen, Vaishnav Tadiparthi
SynCo is a data synthesis co‑training framework that enables self‑evolving large language models (LLMs) to improve autonomously. It employs two agents—a Synthesizer that generates training tasks based on the Reasoner’s evolving capabilities, and a Reasoner that learns from these tasks—using multi‑agent reinforcement learning to jointly optimize task quality and model performance. Experiments on eight mathematical reasoning benchmarks show that SynCo outperforms existing synthetic‑data methods, especially on previously unsolved problems.
By Wei Yang, Shawn Li, Yuehan Qin, Yawei Wang, Mingxi Wang, Shixuan Li, Tiankai Yang, Jiate Li, Jesse Thomason, Xuezhe Ma, Yue Zhao
The paper introduces SELF, a framework that jointly optimizes environmental feedback modeling and hindsight self-distillation for language agents in reinforcement learning settings where rewards are unavailable. SELF predicts environmental responses while distilling guidance from a feedback-conditioned self-teacher into the policy, creating a mutually reinforcing mechanism that strengthens both supervision and policy learning. Experiments with Qwen3-8B show that SELF outperforms SDPO and GRPO on $ au$-bench and AppWorld tasks, improving success rates and goal completion.
By Hangxi Guo, Fengyuan Liu, Yue Wang, Yuhua Qi, Haoyi Xiong, Fei Sun, Mengnan Du
Fed-GRPO is a federated training framework for Large Language Models that extends Group Relative Policy Optimization (GRPO) to operate without sharing raw data. It introduces three reward‑signal‑driven mechanisms—signal‑weighted aggregation, global reward calibration, and adaptive sparse communication—to guide client updates based on reward statistics. Experiments on mathematical reasoning tasks show that Fed‑GRPO outperforms other federated methods, approaches centralized performance, and reduces communication by up to 32× while allowing extreme compression with minimal accuracy loss.
By Pengxin Guo, Shuang Zeng, Zonggen Li, Weiying Zheng, Mengting Liu, Liangqiong Qu
RoboAware learns to coordinate modular robot skills by training a state‑conditioned responsibility coordinator using counterfactual outcomes. It introduces State‑Locked Counterfactual Branching (SCB) to generate outcomes for all admissible skill families from the same state, and Execution‑Aware Learning (EAL) to distill these outcomes into family‑conditioned values via Monte Carlo tree search and Q‑learning. In single‑episode evaluations across 100 tasks, RoboAware achieves a 77.0% overall success rate, outperforming existing baselines on RoboSuite, LIBERO‑Pro, and RoboTwin bimanual tasks.
By Bohan Zhou, Xingbei Chen, Emily Huang, Weilin Ruan, Haojian Huang, Yehang Zhang, Zexi Li, Wenqian Li, Qize Yu, Zetian Song, Leyi Wu, Jinghao Li, Mingxuan Song, Xinrun Xu, Zongyang Qiu, Yangkai Wei, Tianyi Zhang, Kaiwen Zhou, Yinchuan Li, James Cheng
VibeEdit is a new image‑editing tool that lets users draw spatial marks and add brief notes directly on an image to create a canvas instruction. These instructions guide the model to perform tasks such as adding, removing, replacing, or moving objects, as well as modifying attributes, without needing a separate text prompt. The system is trained on 1.55 million annotated edit pairs and uses a layer‑decoupled conditioning approach, achieving higher VLM rubric scores and PSNR on a benchmark that tests target selection among similar objects.
By Jinjing Zhao, Fangyun Wei, Yitong Wang, Xiuyu Wu, Yunuo Chen, Yang Yue, Sirui Zhang, Wenbo Wang, Hongyang Zhang, Dong Chen, Yan Lu, Chang Xu
The paper introduces the World-Model Policy Arbiter (WMPA), a test‑time framework that selects among a portfolio of frozen goal‑conditioned policies by simulating each policy’s future trajectories in a learned world model and evaluating them with a shared value function. WMPA avoids frequent policy switching by committing to the chosen policy for a short interval and does not require retraining or task‑specific knowledge. On the OGBench benchmark, WMPA raises the macro‑average success rate from 44% to 58%, achieving significant improvements on 12 of 18 datasets, including large gains on cube‑double‑play and scene‑play.
By Junwei Quan, Evgenii Opryshko, Nicholas Rhinehart, Igor Gilitschenski
The paper presents a preference-driven unlearning framework for few-step distilled text-to-image diffusion models. It identifies that standard Direct Preference Optimization (DPO) fails on these models due to altered generation dynamics and proposes a modified formulation aligned with few-step properties. The method enables direct concept removal while preserving desirable capabilities, and is evaluated on identity, NSFW, and object-level unlearning tasks with strong forgetting and retention results.
By Gaurav Patel, Jun Fang, Greg Ver Steeg, Qiang Qiu, Sravan Sripada
The paper introduces Success Guided Sampling (SGS), an adaptive sampler that focuses reinforcement learning on task configurations near the policy’s current frontier, thereby reducing wasted experience in massively parallel simulation. SGS enables large‑scale RL to solve complex quadruped locomotion and contact‑rich assembly tasks that previous methods could not, using up to one million parallel environments. The authors also distill the learned policies into RGB‑based models that transfer zero‑shot to real‑world assembly tasks.
By Octi Zhang, Mateo Guaman Castro, Patrick Yin, Ignacio Dagnino, Abhishek Gupta, Rosario Scalise, Byron Boots