Dex-One2Many introduces a real-to-sim-to-real framework that learns a generalizable dexterous manipulation policy from a single human video. By abstracting the video into sequential scene graphs, the method guides reinforcement learning with dense rewards and diverse reset states, enabling efficient exploration while preserving broad generalizability. Trained entirely in simulation, Dex-One2Many achieves zero‑shot transfer to a real multi‑fingered hand and outperforms baselines by 6.5% on seen configurations and 71% on unseen scenarios across five tool‑use and manipulation tasks.
By Jusuk Lee, Sungha Kim, Yeonsoo Park, Jonguk Cheon, Yoonkyo Jung, Yongjun You, H. Jin Kim, Jia-Bin Huang, Furong Huang, Youngseok Jang, Seungjae Lee
The paper introduces Entropy‑Normalized Trust Region (ENTR), a method for asynchronous reinforcement learning that adjusts off‑policy ratio thresholds based on token entropy rather than a single magnitude cut‑off. By recognizing that the natural scale of the ratio is set by entropy, ENTR preserves genuine exploration while filtering out noise from low‑entropy, stale data. Experiments on long‑horizon agentic tasks and mathematical reasoning benchmarks show ENTR outperforms existing asynchronous methods, improving BrowseComp‑Plus performance by 6.9 % and enabling stable training up to 30 policy versions of staleness while matching synchronous GRPO at a 2.6× speedup.
By Guanqun Zhao, Zijun Xie, Binbin Zheng, Yehan Yang, Jiafeng Lu, Aoqi Hu, Enlei Gong, Zeyu Chen
Agentic Critical Training (ACT) is a reinforcement learning approach that trains language‑model agents to judge actions directly by pairing expert actions with alternatives and rewarding correct selection. Unlike imitation learning or self‑reflection, ACT does not require reference rationales and can be reused across model sizes. Experiments on Qwen3‑8B and Olmo‑3‑7B‑Instruct show that ACT improves performance on ALFWorld, WebShop, ScienceWorld, and outperforms supervised reflection baselines, with significant gains over IL and IL→RL baselines.
By Weize Liu, Minghui Liu, Sy-Tuyen Ho, Yongkyun Lee, Andrew Adams Schoen, Souradip Chakraborty, Xiyao Wang, Furong Huang
The paper addresses reward hacking that arises when large language models (LLMs) are post‑trained using cheap surrogate judges instead of expensive human feedback. It introduces envelope sampling, a theoretically grounded recalibration technique that uses a small set of ground‑truth annotated outputs to adjust the surrogate judge before further training. Experiments on clinical note generation and a sycophancy task demonstrate that envelope sampling reduces reward hacking compared to traditional base‑model recalibration methods.
By Sanjit Dandapanthula, Shuvom Sadhuka, Samir Khan, Michael Oberst, Aaditya Ramdas, Alexandra Chouldechova
The paper investigates the use of Jev, a decision model that produces calibrated, typed answers in a single forward pass, as a component in reinforcement learning (RL). It shows that Jev can satisfy most requirements of RL systems except providing a value function, and proposes algorithms that use Jev as a reference policy, exploration judge, and replay rater. Experiments on MiniGrid and Atari tasks demonstrate that incorporating Jev improves sample efficiency, exploration, and learning performance compared to a standard RL learner, even when the learner alone fails to progress.
By Yi Ma, Tianpei Yang, Yaodong Yang, Weixun Wang, Hongyao Tang
MA-JEPA is a stochastic world model designed for multi‑agent reinforcement learning that replaces observation reconstruction with joint‑embedding prediction. It uses a categorical latent state and a causal Transformer trained on posterior and action‑conditioned dynamics to generate imagined trajectories for actor‑critic learning. The model achieves strong performance on the SMAC benchmark, matching or surpassing the best reported mean win rates on half of the evaluated maps.
By Brandon Gary Kaplowitz, Osaze James Obahor, Christian Schroeder de Witt
The paper investigates the sample‑efficiency of Kolmogorov‑Arnold Networks (KANs) in reinforcement learning. Using experiments on the Feynman dataset and Gymnasium benchmarks, the authors find that KANs achieve comparable performance to standard Multi‑Layer Perceptrons while requiring 40% fewer samples, with up to 50% relative performance gains during training. These improvements hold even when reward noise varies, underscoring KANs’ potential for more sample‑efficient RL.
By Kevin Riehl, Shaimaa K. El-Baklish, Fan Wu, Anastasios Kouvelas
DaCe-DT is an offline multi‑task reinforcement learning framework that tackles data‑level bottlenecks by introducing length‑gated prompt masking, retrieval‑augmented prompt construction, and value‑adaptive return calibration. These mechanisms adapt prompts to diverse task complexities, ensure semantic relevance of prompt segments, and correct misleading supervision from fragmented trajectories. Experiments on Meta‑World show that DaCe‑DT outperforms state‑of‑the‑art methods, improving performance by 11.73% on optimal datasets and 13.34% on suboptimal datasets.
By Xinfei Wang, Shanchen Pang, Chenhao Zhang, Shudong Wang, Wenhao Ji, Haiyuan Gui, Meng Han, Xiaojian Liao
The article discusses unsupervised machine learning, noting it differs from supervised and reinforcement learning by being a heterogeneous field with multiple objectives. It argues that defining a single goal for unsupervised learning is futile and identifies four distinct goals: estimating the distribution, generating new data points, extracting features for downstream tasks, and understanding the data.
By Aapo Hyv\"arinen
The paper introduces Adversarial Heuristic Learning (AHL), a framework that uses AI agents to refine game policies without changing model weights. It presents AAArena, a benchmark of 12 adversarial games and 1,920 human programs, where agents analyze replays, select opponents, and revise policies to maximize ranking within limited budgets. Experiments show that certain configurations achieve gold medals, while performance drops in games with complex rules, highlighting challenges in game understanding and long‑horizon policy development.
By Kaisen Yang, Qingle Liu, Kejin Wang, Yicheng Zhao, Jieming Li, Shenghan Zheng, Ruize Yang, Bojun Yang, Heng Gong, Xiang Gao, Lanyue Zhang, Kaiyu Zhong, Zhuo Liu, Shaoxuan Li, Chengxi Li, Yong Yan, Weixuan Zhang, Tianwei Luo, Situ Wang, Youjie Zheng, Sihan Zhao, Shengyuan Wang, Huan-ang Gao, Jiazheng Xu, Xiaohui Xie, Wentao Han, Hongning Wang
The paper introduces a new Bellman operator that merges performance and safety objectives into a single joint value function for reinforcement learning. Temporal difference learning with this operator converges under a two‑timescale stochastic approximation, estimating safety on a fast timescale and joint value on a slow one. Theoretical analysis shows convergence to safety‑constrained optimal policies, and experiments on continuous control tasks confirm stable learning with near‑zero safety violations.
By Nishanth Arun Rao, Royina Karegoudra Jayanth, Benjamin Eysenbach, Jaime Fern\'andez Fisac
OneSearch‑VL is a unified multimodal deep research agent that handles single‑image, multi‑image, and video tasks by leveraging a Visually Grounded Evidence Graph (VGEG) to encode dependencies among visual anchors, entity relations, facts, and answer‑producing operations. The VGEG‑based data engine builds and verifies questions, filters expert trajectories, and supports supervised fine‑tuning (SFT) and reinforcement learning (RL) with the Evidence‑aware Visual‑Grounded Rubric reward (EVGR). Experiments demonstrate that OneSearch‑VL‑8B outperforms Qwen3‑VL‑8B with tool access by 20.2 and 17.6 percentage points on new multimodal benchmarks, while also achieving significant gains across seven image benchmarks and VideoDR.
By Hongyu Li, Manyuan Zhang, Kaituo Feng, Shu Chen, Dian Zheng, Hao Li, Hao Yu, Zhangquan Chen, Zoey Guo, Ray Zhang, Shaofei Huang, Tianrui Hui, Linjiang Huang, Si Liu
Rubric-CEPR is a self‑evolving image editing framework that improves a pretrained editor using only its own generated samples, without human‑edited targets or external reward models. It employs a planner to generate structured edit instructions, an editor to produce multiple candidate edits, and a frozen critic that scores each candidate with rubric‑augmented contrastive edit‑preservation rewards, rejecting infeasible candidates via non‑compensatory gates. The best verified candidate is distilled into the editor through lightweight adapter training, yielding measurable performance gains on benchmarks such as Qwen‑Image‑Edit, GEdit‑Bench, and Complex‑Edit.
By Ritesh Thawkar, Shubham Patle, Shravan Venkatraman, Rao Muhammad Anwer
DreamTrue is a multi‑view, cross‑embodiment robot world model that predicts physically plausible videos while faithfully following actions. It overcomes calibration and limited interaction coverage issues by rendering action trajectories into image‑space conditions, applying offline geometric calibration, and using counterfactual post‑training to generate a broader range of future videos. A human‑annotated defect dataset trains a video reward model that guides reinforcement‑learning post‑training, resulting in state‑of‑the‑art action following on AgiBot and a significant drop in interaction defect rates.
By Junyan Li, Ruizhi Li, Yu Liu, Xiangshuo Liu, Mingchao Sun, Hongyu Pan, Mu Xu, Lue Fan, Zhaoxiang Zhang
The paper explores optimal trading under partial information, modeling the trading signal as an Ornstein‑Uhlenbeck process with regime switching. It proposes three reinforcement‑learning approaches that integrate recurrent neural networks to filter latent market regimes, comparing a one‑step method and two two‑step methods that use posterior regime probabilities or signal forecasts. Simulations and an equity pairs‑trading case study show that using posterior probability estimates of the latent regime yields the best cumulative rewards and more interpretable strategies.
By Andrea Macr\`i, Sebastian Jaimungal, Fabrizio Lillo
The paper introduces two distillation methods for handling misaligned AI models. Distillation for Incrimination (DFI) transfers misalignment without the ability to conceal it, making students more likely to admit hidden behaviors. Distillation for Capabilities (DFC) transfers useful capabilities while minimizing misalignment, using techniques like inoculation prompting and focused training.
By Sebastian Prasanna, Jacqueline Tay, Alek Westover
The paper investigates how incorporating higher‑order mechanical structure into reinforcement learning policies can improve quadruped locomotion when actuators degrade. By modeling the Unitree Go1 robot as a cell complex and using Hodge‑based message passing across node‑edge‑face relationships, the authors show that this higher‑order morphology actor outperforms other approaches on unseen actuator degradations, achieving higher returns, better survival, and lower velocity‑tracking error. The study demonstrates that explicitly encoding complex mechanical relationships provides a powerful inductive bias for whole‑body compensation in degraded systems.
By Derek You, Zafir Shamsi, Keqin Wang, Christine Allen-Blanchette
Large language model agents often reason before every action, but this may be unnecessary if earlier reasoning still supports later actions. The paper introduces RACE, a method that uses likelihood-based signals to detect when reasoning can be omitted, called LoGiC. Experiments on four benchmarks show RACE cuts reasoning cost while keeping or improving performance.
By Yiruo Cheng, Shen Huang, Xiaoshuai Song, Jiejun Tan, Guanting Dong, Pengjun Xie, Ji-Rong Wen, Zhicheng Dou
The paper proposes an adaptive multi‑discriminator Wasserstein GAN (MD‑WGAN) framework for Internet of Vehicles (IoV) that combines reinforcement learning and game‑theoretic coordination to manage resource allocation for distributed training. Roadside units host generators with Deep Q‑Network agents that select discriminator subsets and coordinate training epochs, while a unified objective links adversarial quality to resource efficiency, communication overhead, and latency. Experiments on real‑world NGSIM trajectory data show comparable prediction accuracy to state‑of‑the‑art GANs, with a 28% reduction in CPU usage and 6% reduction in memory usage, while maintaining competitive communication overhead and latency.
By Farhoud Jafari Kaleibar, Amr M. Zaki, Marin Litoiu
The paper introduces PFN‑OPE, a prior‑data fitted network that performs amortized off‑policy evaluation (OPE) for large language models (LLMs). Unlike traditional OPE methods that must be retrained for each new logged dataset or reward definition, PFN‑OPE is pretrained once on a diverse set of contextual‑bandit tasks and can estimate value for any new logged dataset and target response in a single forward pass. Experiments on HelpSteer2 and UltraFeedback with Qwen, Llama, and Gemma policies show that PFN‑OPE reduces error by 2.0 to 9.3 times compared to the best baselines in reward‑shifted settings.
By Younwoo Choi, Leo Feng, Vincent Liu, Haanvid Lee