The paper proposes a fairness‑guided deep Q‑network (DQN) to control the transmission opportunity (TXOP) of NR‑U in unlicensed spectrum, aiming to balance coexistence with Wi‑Fi. Two reward designs, Q1 and Q2, target different fairness regimes: Q1 enforces strict per‑node throughput balance, while Q2 allows a broader fairness band and rewards throughput gains within it. The DQN policies are benchmarked against 3GPP LBT, a multi‑armed‑bandit approach, utility‑fairness tabular Q‑learning, and a fixed‑TXOP baseline, with experiments across four priority classes and asymmetric populations measuring aggregate throughput, Jain fairness, and concave throughput utility.
By Po-Heng Chou, Yi-Fang Yu, Shou-Yu Chen, Chiapin Wang
The paper introduces Cross-Behavior Policy Optimization (CBPO), a method that unifies offline demonstrations and policy-generated rollouts into a single regression loss using a fixed-reference target. CBPO eliminates the need for separate supervised fine‑tuning and reinforcement learning objectives, allowing positive and negative offline completions, reference responses, and current and historical rollouts to share a common reward‑labeled format and replay buffer. Experiments on two model scales and four mathematical reasoning benchmarks show that CBPO, especially the CBPO‑Unified variant, outperforms baseline methods in avg@32 and pass@32 metrics, with historical rollouts providing additional gains.
By Du Cheng, Junyu Lu, Jingfan Zhang, Yuzhuo Zhang, Haoxuan Jia, Shichao Weng
The paper introduces Golden Anchor Learning (GAL), a framework that optimizes text-to-image (T2I) generators using feedback from downstream image‑to‑video (I2V) models to produce ‘golden anchors’ that improve video quality. Experiments show that GAL consistently raises VBench scores by 2.97%–8.46% and enhances motion quality across four I2V models, even when trained only on HunyuanVideo‑1.5 renders.
By Jinyuan Xu, Shitong Shao, Lichen Bai, Zikai Zhou, Guoxi Huang, Nantheera Anantrasirichai, Zeke Xie
The paper introduces a reinforcement‑learning framework for managing bundles and flow control in deep‑space networks, where relay nodes must schedule and drop data under intermittent connectivity and limited storage. It defines a goal‑oriented loss based on excess age to prioritize mission‑relevant information and formulates relay control as a constrained semi‑Markov decision process solved with masked Lagrangian PPO. Simulations show that the approach improves age‑based utility and inter‑flow fairness compared to traditional schedulers.
By Sajjad Baghaeea, Aimin Lib, Shanmugarajan Balasubramanian, Elif Uysal
The paper introduces RL‑Kirigami, an inverse design framework that uses optimal‑transport conditional flow matching (OT‑CFM) and reinforcement learning to generate compatible ratio fields for compact reconfigurable parallelogram quad kirigami. A marching decoder enforces global geometric compatibility, while Group Relative Policy Optimization (GRPO) aligns the generator with nondifferentiable rewards for silhouette matching, feasibility, and ratio‑field regularity. The method achieves high silhouette IoU scores (up to 94.91%) and reduces forward simulator evaluations from hundreds to one, enabling rapid laser‑cut prototypes in polymeric sheets within about eight minutes.
By Milad Yazdani, Shahriar Shalileh, Dena Shahriari
arXiv:2609. 38880v2 Announce Type: replace-cross Abstract: We study the last iterate of standard importance-weighted temporal-difference (TD) learning from a single trajectory of a finite Markov decision process.
By Yang Peng
SimVTA is a post‑training framework that enhances driving world action models by aligning simulated future videos with candidate trajectories. It samples trajectories, renders high‑reward futures in a real‑world simulator using 3D Gaussian Splatting, and prioritizes video supervision for high‑error futures. The approach improves planning performance, raising PDMS from 86.2 to 91.3 on NAVSIM v1 and achieving strong results on NAVSIM v2 and HUGSIM.
By Changhui Jing, Haochen Tian, Hongchen Li, Tianyu Li, Caojun Wang, Zihao Zhang, Yanhao Liu, Chengen Xie, Bailin Li, Hao Zhao, Zhenjie Yang
The paper introduces SWEAR, a framework for audio-language models that uses a five‑dimensional difficulty score to guide reasoning. By learning difficulty signals during fine‑tuning and applying reinforcement learning, SWEAR allocates reasoning effort adaptively across stages. Experiments on MMAU‑test‑mini and MMAR demonstrate that SWEAR boosts accuracy while cutting the number of generated tokens, especially for easier cases.
By Yixuan Li, Minghao Lv, Xuenan Xu, Zihao Zheng, Mengyue Wu
AlphaNIST demonstrates that a single reinforcement‑learning policy can master 20 diverse assembly tasks on the NIST Assembly Task Board #1 using only a sparse binary reward, without demonstrations, progress shaping, scripted primitives, or per‑task tuning. The policy achieves 98.2% overall training success and can complete ten assembly pairs in a single episode, discovering dexterous, planning‑like behaviors such as selecting future grasps and reorienting parts using surrounding geometry. The work shows that task‑agnostic rewards can enable robots to learn complex manipulation strategies autonomously.
By Octi Zhang, Iretiayo Akinola, Rosario Scalise, Mateo Guaman Castro, Patrick Yin, Tobias Widmer, Miles Macklin, Chris Amevor, Dylan Turpin, Dirk Van Gelder, Jan Carius, Ankur Handa, Gavriel State, Yunrong Guo, Yashraj Narang
The paper introduces MENTOR, a multi‑agent framework that provides on‑policy pedagogical intervention to address zero‑reward collapse in reinforcement learning from verifiable rewards. MENTOR diagnoses failures in real time and generates targeted prompts to guide the learning agent, integrating seamlessly with existing policy optimization pipelines such as GRPO. Experiments on math and code generation tasks show that MENTOR significantly reduces collapse, achieving up to 78.3% relative gains on Llama‑3.2‑3B‑Instruct.
By Guanlan Yue, Hang Yu, Xiaoyong Huang, Ruipeng Li, Wei Zhang, Bingchang Liu, Rui Wang
The paper examines on‑policy self‑distillation, where a teacher scores a student’s rollout to provide dense token‑level credit. Experiments show that the raw scoring signals neither localize errors nor effectively penalize incorrect tokens, because locating errors is a search problem that the teacher is not explicitly asked to perform. Introducing a critique‑conditioned approach—first generating a critique of the rollout and then conditioning the teacher on it—produces signals that accurately localize errors, correct erroneous spans, and guide the policy toward backtracking tokens, improving performance on math problems and in cold‑start GRPO settings.
By Mayug Maniparambil, Ayush Rai, Ekansh Verma, Noel E. O'Connor, Fergal Reid
The paper presents an unsupervised multimodal reward model built on a frozen Qwen3‑VL backbone. By adding a linear head to an intermediate layer and training it to rank a web caption higher than a perturbed version of itself, the authors avoid human preference labels and achieve competitive performance—outperforming GPT‑4o on several benchmarks with far fewer parameters and compute. The approach demonstrates that a carefully chosen intermediate layer, simple perturbation-based training, and verifier‑driven fine‑tuning can yield effective vision‑language rewards without costly annotation.
By Yueying Li, Jingxuan Fan, Kiant\'e Brantley, Sham M. Kakade, Hanlin Zhang
JoyInside‑VLM‑Agent is a ReAct‑style multimodal agent that learns product‑conditioned policies for embodied robots and smart hardware. It uses a closed‑loop sandbox to generate interaction trajectories, supervised fine‑tuning, multi‑teacher on‑policy distillation, and reinforcement learning with DPER to align token‑level evidence and rewards. The agent achieves a BFCL v4 overall score of 77.44, surpassing baselines on JoyInside‑Devices and JoyInside‑Robotics, and demonstrates visually grounded search, replanning, and failure reporting in two real‑world quadruped case studies.
By Gangyi Li, LuYao Wang, Jingfu Sun, Jingyi Yuan, Hang Liu, Hao Zong, Yujia Hu, Tingting Gao, Jianyong Wang, Xu Yang
MotionCoach is a multimodal agent designed to generate precise guidance for professional motion editing in videos. It constructs grounded prompts and reference clips through multi‑turn tool use and self‑verification, leveraging a training pipeline that includes a 10k SFT dataset and a 2k RL dataset optimized with Group Relative Policy Optimization. Evaluation on the VPMEBench benchmark shows significant performance gains, raising overall scores from 55.1 to 65.6 on Seedance 2.0 and from 51.2 to 64.5 on MiniMax‑H3.
By Siyi Qian, Hanqing Yang, Weihuang Lin, Zihao Yang, Yichen Lu, Yuanhong Zheng, Sheng Li
The paper investigates whether a frozen recurrent network can provide sufficient memory for partially observable Markov decision processes (POMDPs). Using a fixed recurrent network combined with a trainable output module, the authors demonstrate that such networks can recall information over hundreds of steps and achieve competitive control performance across diverse POMDP tasks, while using fewer than half the trainable parameters of fully trainable recurrent networks. These findings suggest that learning the recurrent memory module is not always necessary, offering a simple, parameter‑efficient baseline for partially observable reinforcement learning.
By Zekang Wang, Zhe He, Gaojie Jin
MAGE (Multi-Agent Graph-guided Evolution) is a framework that externalizes self-knowledge of language-model agents into a four-subgraph co-evolutionary knowledge graph, allowing a frozen execution model to retrieve task-conditioned guidance from past reasoning traces and teacher-written corrections. During evolution, the graph, a task-level search bandit, and a skill-level routing bandit are updated from the same reward stream while the learner’s backbone remains unchanged. Across nine diverse benchmarks, MAGE outperforms prompt-based frozen-backbone baselines, and ablations show that self-harvested success traces and teacher-written corrections complement each other, with success memories aiding reasoning-template-heavy tasks and corrective memories supporting harder composition and interaction settings.
By Ruiyi Yang, Zechen Li, Hao Xue, Imran Razzak, Flora D. Salim
The paper introduces RobustBench‑TC, a benchmark that tests tool‑use language agents under 22 real‑world perturbations affecting observations, actions, rewards, and transitions. Experiments on 21 models show that observation noise has minimal impact, while reward‑relevant and transition noise drastically reduce accuracy, and model size alone does not mitigate these gaps. The authors propose ToolRL‑DR, a domain‑randomized reinforcement‑learning recipe that improves robustness, retaining most clean accuracy and narrowing performance gaps even on unseen transition perturbations.
By Xiaolin Zhou, Aojie Yuan, Zheng Luo, Zipeng Ling, Xixiao Pan, Yicheng Gao, Haiyue Zhang, Jiate Li, Shuli Jiang, Prince Zizhuang Wang, Zixuan Zhu, Jinbo Liu, Ryan A. Rossi, Hua Wei, Xiyang Hu
The paper extends theoretical guarantees for offline reinforcement learning to parameterized policy classes over large or continuous action spaces, overcoming limitations of prior algorithms that required finite action sets and state‑wise mirror descent. By addressing contextual coupling and linking mirror descent to natural policy gradients, the authors provide new analyses, algorithmic insights, and a surprising connection between offline RL and imitation learning.
By Xiang Li, Yuheng Zhang, Nan Jiang
AnomalyAgent is an agentic framework for industrial anomaly synthesis that uses five tools—Prompt Generation, Image Generation, Quality Evaluation, Knowledge Retrieval, and Mask Generation—to iteratively refine anomaly samples. It employs a two‑stage training process combining supervised fine‑tuning and reinforcement learning, guided by a three‑part reward system that encourages task performance, self‑reflection, and trajectory adherence. On the MVTec‑AD dataset, AnomalyAgent outperforms zero‑shot state‑of‑the‑art methods, achieving superior anomaly generation scores and high classification and detection accuracy.
By Jiaming Su, Tengchao Yang, Ruikang Zhang, Zhengan Yan, Haoyu Sun, Linfeng Zhang
The paper introduces Stochastic MeanFlow Policies (SMFP), a one‑step generative policy class that merges MeanFlow noise‑to‑action mappings with Gaussian reparameterisation. SMFP captures diverse action modes while providing an analytic entropy lower bound through a learnable conditional Gaussian scale, enabling entropy‑regularised mirror‑descent learning. Experiments on seven MuJoCo benchmarks show SMFP outperforming Gaussian and generative baselines, and additional tests demonstrate effective multimodal action modelling and exploration in sparse‑reward navigation.
By Zeyuan Wang, Da Li, Yulin Chen, Yuehu Gong, Yanming Guo, Liang Bai, Tianyuan Yu, Yanwei Fu