The paper introduces Fast Regularized Policy Mirror Descent (PMD) that pairs policy updates with a single temporal-difference (TD) critic step. It proves global linear convergence for finite discounted MDPs using exact coordinate-wise Bellman updates and any positive actor stepsize, regardless of critic initialization. For stochastic TD-PMD with strongly convex mirror maps, the authors achieve an expected value gap of ε after “~O(1/((1-γ)^5 σ_b ε))” transitions, without requiring trajectory resets, generative models, or nested policy evaluation loops.
By Qipei Chen, Wenye Li, Yule Sun, Ke Wei
The paper introduces Fork-dLLM, a hybrid sampling strategy for diffusion language models that combines parallel token generation with autoregressive ordering only at uncertain fallback steps, thereby avoiding the flexibility trap that reduces generation diversity. It also presents ForkGRPO, an extension that applies the GRPO objective selectively during post‑training rollouts, preserving policy‑likelihood ratios while cutting rollout and optimization costs. Experiments show Fork-dLLM achieves pass@k scaling comparable to autoregressive sampling but 2–3× more efficient, and ForkGRPO matches or surpasses AR‑based GRPO baselines at significantly lower training expense.
By Stipe Frkovi\'c, Metod Jazbec, Christian A. Naesseth
The paper presents C2C (From Codebase to Culprit), a bug‑localization framework that narrows the debugging search space through semantic retrieval and hierarchical reinforcement learning. It first retrieves likely buggy files using contrastively fine‑tuned CodeBERT embeddings and then progressively localizes bugs from files to functions to lines of code. Experiments on Java and Python datasets show that this multi‑resolution approach improves both retrieval precision and localization accuracy, with ablation studies underscoring the importance of hierarchical decomposition and reward shaping.
By Ankur Garg, Corey Yang-Smith, Rishav Rishav, Ahmad Abdellatif, Samira Ebrahimi Kahou
The paper introduces the Posterior Concentration Phenomenon (PCP), a length‑dependent failure mode where probability‑based rewards collapse to a narrow interval for long reasoning traces, destabilizing verifier‑free reinforcement learning. To address this, the authors propose RLCPR, a framework that uses uncertainty‑aware data sampling and concentration‑aware regularization to mitigate PCP, improving token efficiency and outperforming state‑of‑the‑art baselines on multiple reasoning benchmarks.
By Shiu-Hong Kao, Yubo Zhao, Zhenyu Tian, Pengzhan Sun, Yicong Li, Angela Yao
The article outlines a Ph.D. research agenda aimed at creating provably efficient and practical algorithms for data‑driven sequential decision‑making under uncertainty. It focuses on reinforcement learning and multi‑armed bandits, targeting applications such as recommendation systems, computer networks, video analytics, and large language models. The work seeks to overcome limitations of existing methods—such as reliance on idealized models, lack of robustness to adversarial perturbations, and poor instance‑dependent performance—by developing algorithms that are more efficient, robust, instance‑adaptive, and generalizable to new environments.
By Zhiyong Wang
FutureWorlds is a framework that learns robotic world models by generating alternative future predictions and using them as learning signals. It combines candidate construction, history maintenance, and learning from relative quality through a multimodal discrete autoregressive model and diverse beam search. The MemSPO algorithm further optimizes the world model by converting video trajectory rewards into group-relative advantages, leading to significant improvements in LPIPS scores across RT-1, BridgeV2, and RoboCasa datasets.
By Hao Wu, Shengju Qian, Weiyan Wang, Fan Xu, Fan Zhang, Yuanpeng He, Qingsong Wen, Yuxuan Liang
The paper introduces Dynamic Homing Optimization (DHO), a forward‑process reinforcement learning approach that reinterprets negative‑trajectory optimization as attraction toward positive samples. DHO uses Minimum‑Cost Attractive Matching to assign each negative sample a distinct positive target and Time‑Aware Dynamic Correction to guide trajectories toward those targets. Building on DHO, the authors present Flow3D‑Pro, an image‑to‑3D geometry generation framework that outperforms existing DPO, GRPO, and NFT‑style objectives and produces higher‑quality 3D meshes.
By Zhen Zhou, Zhiwei Ning, Puhua Jiang, Sheng Zhang, Yifei Tang, Jie Yang, Xintong Han, Wei Liu, Chunchao Guo
The paper introduces Adaptive Tversky Policy Optimization (ATPO), a reinforcement learning framework for multi‑label video safety detection. ATPO uses an Adaptive Tversky Reward (ATR) that dynamically adjusts false‑positive and false‑negative penalties, allowing controllable precision‑recall trade‑offs. Experiments on SafeWatch‑Bench and XD‑Violence demonstrate significant performance gains, raising the Jaccard Index from 40.66 to 75.44 on SafeWatch‑Bench‑Real.
By Guangyu Yang, Jingbiao Mei, Mingsheng Sun, Jinghong Chen, Yingtong Bu, Pengda Qin, Da Chen, Bill Byrne
The paper introduces a new algorithm for a nonstationary bandit setting where actions influence both immediate rewards and the evolution of an unobserved latent linear state. By approximating the infinite‑memory reward process with a finite‑memory block‑level proxy and applying a UCB‑based block algorithm, the authors achieve a regret bound of “~O(√T)”, improving upon the previous “~O(T^{2/3})” guarantee. This represents the first such “~O(√T)” result for latent linear‑dynamics bandits with bilinear rewards and an open‑loop action‑sequence benchmark.
By Taehyun Hwang, Hyunjun Choi, Heesang Ann, Min-hwan Oh
IQS-BO introduces an in‑context query selection method for Bayesian optimisation that learns to choose evaluation points via supervised learning on synthetic priors. The approach uses a Prior‑data Fitted Network (PFN) to predict, in a single forward pass, the probability that each candidate maximises the objective, thereby amortising the decision step and eliminating the need for costly surrogate refitting or acquisition maximisation. Experiments show that IQS‑BO matches or surpasses traditional Gaussian‑process‑based BO and other in‑context methods on both synthetic and real‑world benchmarks, and that a mixture prior combining GP samples with challenging function types further improves performance.
By Luca Geminiani, Nadja Klein
The paper introduces minimum attention, a principle based on the least action concept, into reinforcement learning rewards. It explores model‑based meta‑learning with this principle in high‑dimensional nonlinear dynamics, alternating ensemble‑based model learning with gradient‑based meta‑policy learning. Experiments show that minimum attention enhances fast adaptation, reduces variance from model and environment perturbations, and improves energy efficiency compared to both model‑free and model‑based RL baselines, including modern world‑model frameworks like DreamerV3 and MAMBA.
By Shashank Gupta, Pilhwa Lee
The paper investigates PRM‑Pruned Fragment Grafting (PPFG), an inference‑time technique that extracts high‑reward prefixes from a pruned chain‑of‑thought and grafts them into a sibling decoding process. Experiments on Qwen2.5‑7B‑Instruct with Math‑Shepherd across 500 MATH problems and multiple seeds show that PPFG performs statistically indistinguishable from a parallel‑CoT baseline, with only 14% of grafts targeting genuinely struggling chains. The study extends across three language models, six benchmarks, and multiple PRM configurations, concluding that PPFG’s inertness is not due to heuristic specifics and providing an equivalence‑testing framework for mechanism nulls.
By Khawaja Murad ul Hassan, Mehran Ebrahimi
The paper introduces a meta-multi-agent reinforcement learning (meta‑MARL) framework that enables rapid adaptation of interactive policies in multi‑agent systems. By modeling multi‑agent reinforcement learning problems as Markov games and defining a new concept called meta‑NE, the authors establish conditions linking meta‑NE to stationary points of a gradient‑play meta‑MARL algorithm. Experiments on autonomous‑driving tasks show that this approach adapts faster than pretrained MARL baselines, demonstrating its effectiveness.
By Huiwen Yan, Kyriakos G. Vamvoudakis, Mushuang Liu
The paper introduces Generalized Implicit Temporal Abstraction (GITA), a method for goal-conditioned reinforcement learning that conditions a single value function on multiple temporal abstraction levels (k). By aggregating advantage-weighted supervision across various k values, GITA preserves both long-range signal and local resolution without committing to a single k. Experiments on OGBench show that GITA outperforms existing offline GCRL baselines, improving average success rates by 25 percentage points over HIQL and 7 percentage points over OTA.
By Pedro Robles Dutenhefner, Dikshant Shehmar, Wagner Meira Jr., Marlos C. Machado
ActiveSaddler introduces automated curriculum learning for harness optimization, treating the evolving training curriculum as a non‑stationary bandit problem. It identifies reusable failure patterns, estimates learning progress for each, and balances revisiting known weaknesses with exploring new scenarios, allowing the curriculum to co‑evolve with the harness. Experiments on GAIA2 and Terminal‑Bench 2.0 show consistent improvements in harness performance, with Pass@1 gains of 4.4 and 7.5 percentage points over fixed‑order baselines.
By Sungho Park, Wonjoong Kim, Jue Zhang, Wook-Shin Han, Pengfei Gao, Chanyoung Park, Yongqiang Yao, Rao Fu, Elsie Nallipogu, Qingwei Lin, Victor R\"uhle
The paper introduces APIVIS, a training-time framework that integrates finite-budget Gumbel search into reinforcement learning with verifiable rewards (RLVR) for mathematical reasoning. APIVIS combines direct and searched responses within each rollout group, uses selective supervision on search-improved tokens, and applies value-guided selection to improve verifier rewards at each searched state. Experiments on standard mathematical reasoning benchmarks and various model scales show significant performance gains over existing search-based methods.
By Shaohuai Liu, Yuning Wu, Haoran Liu, Enzo Jia, Devin Chen, Kai Wei
The paper explores how large language models (LLMs) can be grounded in dynamic stochastic general equilibrium (DSGE) simulators to generate and evaluate economic policy actions. By embedding an instruction‑tuned LLM within six Snowdrop‑backed DSGE models, the authors create a long‑horizon credit‑assignment problem where the model observes economic states, selects bounded policy actions, and receives simulated outcomes and rewards. They compare reinforcement learning methods—PPO, which uses a learned value function to propagate delayed rewards, and GRPO, a critic‑free baseline—to assess how well the LLM’s policy decisions align with simulated economic dynamics, testing factors such as semantic signals, reward horizons, and cross‑simulator transfer.
"whyItMatters":"The study demonstrates a method for evaluating LLM‑generated policy proposals based on their actual simulated economic consequences rather than merely their linguistic plausibility."
By Aditya Dubey, Namah Gupta, Vinti Agarwal
LineupRL introduces a reinforcement learning framework with verifiable rewards for time series captioning, using a frozen large language model to identify the correct time series from a set of distractors based on a generated caption. This approach bypasses the limitations of supervised fine‑tuning and traditional RL rewards that poorly transfer to open‑ended time series generation. Experiments on two captioning benchmarks, as well as forecasting and reconstruction tasks, show that LineupRL outperforms both SFT and RL baselines across all metrics, and its trained 3B vision‑language model surpasses a 72B model distilled from SFT captions. The method also demonstrates resistance to reward hacking and produces captions that accurately trace trends and name key values.
By Haochen Zhang, Laura Yao, Zachary Plotkin, Gengwei Zhang, Tianlong Chen
The paper surveys recent advances in Retrieval Augmented Generation (RAG), a technique that integrates external retrieval into language model generation to reduce hallucinations and keep knowledge current. It introduces a four‑axis taxonomy—efficiency, defense, interactivity, and reasoning—to organize contemporary RAG research, covering retrieval methods, fusion strategies, embedding optimizations, and reinforcement learning policies. The survey also reviews evaluation practices, domain‑specific applications, and architectural variants, while highlighting ongoing challenges such as retrieval quality, reliability, domain adaptation, scalability, and explainability.
By Meghana Sunil, Shravya V, Shravan Venkatraman, Joe Dhanith PR
SPHERE is an adaptive VR indoor scene generation framework that turns isolated 3D synthesis into continuous human‑AI co‑creation. It learns persistent spatial preferences from multimodal user interactions, abstracts these into hierarchical constraints for geometric resilience, and employs a human‑in‑the‑loop reinforcement learning loop to refine retrieval policies. A mixed‑design study with 42 participants and offline ablation show that SPHERE reduces corrective edits and physical effort while avoiding bias toward shallow object‑level traits, producing geometrically resilient, profile‑aligned layouts.
By Hyeonmin Lee, Zheng Wei, Kyungmin Kwon, Jumin Seo, Jiwon Park, Hayoung Oh