EvoSignal is an LLM‑guided evolutionary framework that treats traffic signal control as a modular program design problem, separating traffic feature extraction, local phase prioritization, and optional network‑based priority adjustment. Starting from established strategies, it iteratively improves programs using congestion and signal operation feedback, ultimately producing control rules that operate without online LLM inference. In simulations on two real‑world road networks, the best EvoSignal program reduced waiting time by 16.8–49.2% compared to 20 conventional, reinforcement‑learning, and LLM‑based baselines, and a travel‑time/queue‑length prioritizing program consistently outperformed all baselines across multiple scenarios.
By Leizhen Wang, Peibo Duan, Zhenlin Qin, Yancheng Ling, Jian Xu, Yue Wang, Hao Wang, Zhenliang Ma
The paper introduces COPC, a Coupled Off-Policy Correction method for asynchronous reinforcement learning of large language models. COPC coordinates policy-side and advantage-side corrections by combining token-level ratio masking with two-sided clipped-ratio weighting of TD residuals, addressing both policy mismatch and advantage staleness. Experiments show COPC outperforms existing asynchronous baselines on tool-integrated mathematical reasoning and search tasks, while maintaining training stability and minimal overhead.
By Zicheng Hu, Zhijian Zhou, Xuan Zhang, Yuchen Liu, Cheng Chen, Yuan Li, Qi Gu, Yan Feng, Hongyan Hao, Chao Qu
RollVerify is a lightweight reinforcement learning framework that addresses the trade‑off between efficiency and accuracy in long‑tail rollout settings. It introduces an off‑policy shift metric (OPS) to quantify deviation in partially generated trajectories and uses sequence‑level and token‑level verification to truncate invalid suffixes before training. Experiments on mathematical reasoning tasks show that RollVerify matches on‑policy accuracy while cutting training costs, with preliminary evidence from code‑generation tasks.
By Yongqiang Yao, Jinru Tan, Kaihuan Liang, Zixin Yin, Yazhe Niu, Ruihao Gong, Dahua Lin, Ningyi Xu
The paper introduces Interaction-Constrained Drive Policy (ICDP), an offline reinforcement learning framework for autonomous driving that explicitly controls interaction-level distribution shift. ICDP decomposes joint-support degradation into ego-support and interaction-support components, estimating the latter via contrastive density-ratio estimation without requiring joint-density modeling or simulator rollouts. Experiments on nuPlan, Interplan, and real-world truck data demonstrate that ICDP reduces selection of high-value but interaction-unsupported trajectories and improves performance in interaction-critical driving scenarios.
By Mahmoud Selim, Cristina Cipriani, Karl Henrik Johansson
The paper introduces the expected sample complexity metric for stochastic multi‑armed bandits and analyzes it within a new framework called approximately correct in expectation (ACE). It demonstrates that ACE guarantees lead to almost sure convergence to the optimal expected reward, contrasts this with high‑probability guarantees, and shows how to translate ACE bounds into explicit expected regret bounds. The authors prove that deterministic algorithms cannot achieve favorable ACE bounds, propose an explore‑then‑ε‑greedy algorithm when ε is known, analyze Thompson sampling when ε is unknown, and provide nearly matching lower bounds that establish tightness and a performance separation between the two regimes.
By Nadav Sukenik, Nadav Merlis
The paper introduces a new class of offline adversarial attacks on warm‑start bandit algorithms with bounded rewards. Unlike prior methods that suppress non‑target arms, the authors show that promoting the target arm is essential when it lies near the lower reward boundary, and they design an attack that achieves optimal sublinear cost by balancing target promotion and non‑target suppression. The attack is extended to several popular bandit algorithms and validated experimentally on real‑world and synthetic data.
By Qirun Zeng, Manhin Poon, Xiangxiang Dai, Qixin Zhang, Jinhang Zuo
arXiv:2610. 10273v1 Announce Type: new Abstract: Despite its widespread use, Proximal Policy Optimization with clipping (PPO-Clip) remains difficult to tune, and the interactions among critic learning, clipping, and rollout reuse remain incompletely understood.
By Junwei Su, Mengfan Liu, Yanyong Zhang, Chuan Wu
The paper presents a robotic platform that enables closed‑loop autonomous navigation of liquid droplets on an open surface using model‑based reinforcement learning. A two‑axis tilting board with a silicone oil film moves the droplet while an overhead camera supplies real‑time feedback, and a policy is trained in only 50–150 physical episodes without simulation or analytical models. The system demonstrates zero‑shot transfer to unseen geometries, autonomously discovers an oscillatory depinning strategy to release stuck droplets, and completes the full training pipeline in under 90 minutes.
By Rajneesh Anand, Mayuresh V. Kothare
The paper introduces Q‑Learning with Scalar Adjoint Matching (SQAM), a method that replaces expensive per‑step vector–Jacobian products in flow policy fine‑tuning with a closed‑form scalar adjoint that scales the value gradient by flow time. SQAM also incorporates a value penalty on policy‑generated actions, which is shown to be crucial for effective learning. Experiments demonstrate that SQAM outperforms strong baselines on the hardest OGBench domains and improves supervised fine‑tuning on a real bimanual robot with a vision‑language‑action policy.
By Yonghoon Dong, Minsung Yoon, Jaehyuk Kim, Jungwoo Park, Changyeon Kim, Jinwoo Shin
The paper introduces Joint On-Policy Learning and Teaching (JOLT), a method that trains a single policy to act both as a privileged teacher and an unprivileged student. It derives a condition ensuring the teacher’s distillation update aligns positively with the student’s reward gradient, leading to a KL‑regularized teacher objective that matches the student’s current capabilities. Experiments show JOLT improves training efficiency and performance across various tasks, with additional gains when student rewards are incorporated.
By Randy Ardywibowo, Arnav Dalal, Jiantao Jiao
The paper introduces Counterfactual KKT (CF‑KKT), a framework that learns unknown constraints from locally optimal demonstrations without requiring known dynamics or risky exploration. CF‑KKT uses a learned differentiable dynamics model to apply KKT‑inspired optimality conditions and generates counterfactual behaviors that reveal infeasible data, enabling safer and more data‑efficient constraint recovery. Experiments on high‑dimensional robotic control tasks show that neural constraint representations learned with CF‑KKT achieve better safety and data efficiency than state‑of‑the‑art offline inverse constrained reinforcement learning baselines.
By Zhouyu Zhang, Chih-Yuan Chiu, Glen Chou
arXiv:2610.09858v1 Announce Type: cross
Abstract: Large language model agents tackle multi-step tasks by interleaving reasoning and tool calls with observations from the environment. Prior work has s...
By Sergei Polezhaev, Barys Liskavets, Ori Press, Alexander Golubev
arXiv:2610.09985v1 Announce Type: cross
Abstract: We study a sequential pricing problem in which a seller jointly posts a price and an advertisement generated by a large language model (LLM). The sel...
By Alessandro Barro, Francesco Bacchiocchi, Francesco Emanuele Stradi, Alberto Marchesi
arXiv:2610.10179v1 Announce Type: cross
Abstract: Search agents enable Large Language Models (LLMs) to iteratively retrieve and use information for complex multi-hop questions. Reinforcement Learning...
By Wenyu Huang, Xinyu Hou, Pavlos Vougiouklis, Ruofei Lai, Jeff Z. Pan
arXiv:2610.10297v1 Announce Type: cross
Abstract: While energy efficiency is a critical objective for legged-robot locomotion control, achieving low energy consumption while maintaining robust perfor...
By Ammar Issa, Anubhav Singh, Anton Tsaritsin, Sergey Kolyubin
arXiv:2407.18994v2 Announce Type: replace-cross
Abstract: We consider the automatic online synthesis of black-box test cases from functional requirements specified as automata for reactive implementa...
By Ocan Sankur (DEVINE), Thierry J\'eron (DEVINE), Nicolas Markey (DEVINE), David Mentr\'e (MERCE-France), Reiya Noguchi
arXiv:2606.22108v2 Announce Type: replace-cross
Abstract: Urban traffic congestion remains a persistent challenge in car-dependent cities, imposing significant economic and societal costs. Traffic si...
By Yousef AlSaqabi
arXiv:2610.10426v1 Announce Type: new
Abstract: Terminal-agent capability depends jointly on model weights and the runtime harness that formats prompts, binds tools, and handles error recovery. Exist...
By Jixuan Chen, Jiaxin Zhang, Qinyuan Ye, Yada Pruksachatkun, Haoxiang Zhang, Jingming Zhuo, Yifan Zhang, Yutong Dai, Juntao Tan, Xiangyu Peng, Silvio Savarese, Zeyuan Chen, Lianhui Qin, Chien-Sheng Wu
arXiv:2604.05087v4 Announce Type: replace
Abstract: Generative large language models (LLMs) are increasingly used as inference-time components in retrieval pipelines, for tasks such as query rewritin...
By Omri Uzan, Ron Polonsky, Douwe Kiela, Christopher Potts
arXiv:2609.34798v2 Announce Type: replace
Abstract: Recent medical multimodal models have benefited from larger corpora, broader modality coverage, and stronger reasoning-oriented training, yet effec...
By Guanghao Zhu, Zeyu Liu, Zhitian Hou, Pengkai Wang, Zhijie Sang, Shuo Cai, Yang Yu, Yuanyi Wang, Yanggan Gu, Congkai Xie, Jianmin Wu, Hongxia Yang