The paper investigates how to aggregate preferences in preference‑conditioned reinforcement learning for steerable humanoid control. It evaluates reward‑level, advantage‑level, loss‑level, and PCGrad‑based aggregation methods using Preference Reward Responsiveness, coverage, and utility metrics across two control frameworks. Results show that no single aggregation level dominates; early aggregation yields broader behavioral changes but more unintended effects, while later aggregation offers tighter control at the expense of responsiveness.
By Ziqian Huang, Yuzhuo Chen, Shimin Zhang, Junyang He, Xinyue Annie Yang, Animesh Garg
BIRD‑RL is a new agentic reinforcement learning framework designed to scale training of small language models (SLMs) in data‑centric, stateful environments such as SQL debugging. It introduces a four‑stage curriculum that alternates between improving agent reasoning and trajectory‑adaptive exploration, and relies on a Trajectory‑scoped Persistent Execution Infrastructure to safely reuse database replicas while maintaining trajectory isolation. This approach yields a 12.8× improvement in spatial efficiency and enables 7B and 14B models to achieve competitive success rates on the BIRD‑CRITIC‑SQLite benchmark, matching proprietary agents like Claude‑Opus‑4.6.
By Ge Qu, Jinyang Li, Xiaohan Xu, Xiaolong Li, Bowen Qin, Nan Huo, Ziwei Tang, Chenhao Ma, Reynold Cheng
The paper demonstrates a novel remote Rowhammer attack that targets federated learning (FL) servers by manipulating client sensor observations to induce repetitive clustered memory updates. Using a reinforcement learning attacker, the authors show that they can achieve approximately 70% repeated update rate in FL systems with sparse updates, leading to bit‑flips in the server’s DRAM during tasks such as automatic speech recognition and CIFAR‑10 classification. This work highlights a previously underexplored vulnerability where client‑side adversarial behavior can compromise server memory integrity in FL environments.
By Jinsheng Yuan, Yuhang Hao, Weisi Guo, Yun Wu, Chongyan Gu
TrustAgent‑RFF is a framework that enhances the reliability of radio frequency fingerprinting (RFF) models when faced with domain shifts such as changes in receiver, distance, or environment. It operates on a frozen I/Q classifier and gathers diagnostic evidence—counterfactual stability, prototype consistency, acquisition domain similarity, and signal quality—to estimate prediction reliability. A large language model planner selects diagnostic actions based on evidence, trust state, costs, and budget, while a calibrated decision layer classifies predictions as Accept, Reject, or Defer. Experiments on five RFF datasets show a 38.7% drop in source‑to‑target accuracy and highlight that many incorrect target‑domain predictions remain highly confident.
By Chuan Liu, Tianya Zhao, Yiting Wang, Junqing Zhang, Xuyu Wang
The paper introduces Sibling Disagreement in Imagination (SDI), a method that uses a shared actor-based reference to guide exploration and planning in world models. SDI compares action sequences imagined from the same state, scaling exploration signals and action acceptance based on the variability of the actor’s own rollouts. Experiments on Meta-World, Adroit, and DM‑Control show SDI outperforms baselines on most tasks and improves recovery from actor failures while preserving successful behavior.
By Hieu Tran, Hong Yu
JaxMARL-BC is a JAX‑accelerated simulator that enables multi‑agent reinforcement learning (MARL) for macroeconomic modelling to be trained in seconds, achieving speedups of 75× to 724× over previous implementations and scaling to 200,000 independent learning households. It builds on the MARL‑BC framework, which aligns deep reinforcement learning households with dynamic stochastic general equilibrium (DSGE) models, and verifies its correctness against known DSGE limits. The authors demonstrate the tool’s value by calibrating an extended model with heterogeneous returns on capital in a two‑hour experiment that would otherwise take over a month, and they release the software as open source.
By Daniele Montagnani, Andrea Coletta, Antonietta Mira, Aldo Glielmo
The paper introduces Signals and Queries (S&Q), a lightweight first-stage relevance filter for LLM agents that removes at least 30% of candidate policies before downstream evaluation while maintaining high recall. S&Q uses offline-generated activating examples and hard negatives to train a logistic classifier that predicts policy relevance via embeddings and vector dot products, eliminating per-turn LLM calls. Evaluated on 9,144 relevance labels across 254 conversation prefixes and 36 airline policies, S&Q outperforms top‑K and cosine‑threshold retrieval in precision at every recall level, consistently across two embedding providers.
By Yam Marcovitz
The paper introduces FairAMO, a controlled protocol to evaluate agentic model orchestration (AMO) for time‑series forecasting by freezing candidate forecasts. Experiments on GIFT‑Eval show that while AMO can improve accuracy, its gains are limited by the quality and complementarity of the underlying TSFMs, with reductions in MASE below 6% relative to the best single model. Adding more candidates yields diminishing returns, and redundant members can negate benefits, indicating that historical evidence saturates quickly and that orchestration’s contribution is modest compared to the candidate models themselves.
By Tong Guan, Tianxiang Zhan, Shirui Pan, Ming Jin
ProactEgoMem is a benchmark for evaluating proactive memory in streaming egocentric video, focusing on three tasks—Information Capture, State Tracking, and Location Tracking—to reflect real-world environmental needs. It introduces two levels of proactiveness: Notification Level, which requires a single timely alert, and Intervention Level, which demands continuous memory maintenance via an active tool lifecycle. The benchmark includes daily and scientific datasets with corresponding QA sets, and shows that current models struggle to capture memory both timely and accurately.
By Ruotong Liao, Meikel Kokowski, Xiaoxiao Sun, Yongqin Xian, Laura Bravo S\'anchez, Shiye Su, Federico Tombari, Serena Yeung-Levy, Volker Tresp
This survey reviews over 100 studies on implementing reinforcement learning (RL) on field‑programmable gate arrays (FPGAs) and integrated circuits (ICs), covering a wide range of RL families from model‑based Markov decision processes to deep RL and multi‑agent methods. It organizes the literature by abstraction level, hardware realization strategy, and application domain, and evaluates key metrics such as resource utilization, latency, throughput, and power consumption. The paper identifies common architectural patterns, practical limitations, and open research challenges, offering a unified reference for researchers developing RL‑driven hardware systems.
By Hamza Kheddar
The paper introduces an explicit feedback contract for iterative machine‑learning agents, limiting when agents can observe validation scores and when they may stop iterating. The contract only releases a score once it exceeds a noise threshold derived from the standard error, preventing adaptive over‑selection of the validation split. Experiments on click‑through‑rate prediction, learning‑to‑rank, and a zero‑signal control task show that the contract reduces the validation‑test gap, yields slightly higher test performance, and forces agents to stop earlier—using only 4–12 submissions instead of the full 30 budget used with exact feedback.
By Kehui Yao, Chuanbo Hu, Jiao Chen, Ramin Giahi, Reza Yousefi Maragheh, Lalitesh Morishetti, Ziwei Hong, Yiyun Su, Jianpeng Xu, Evren Korpeoglu, Kannan Achan
SSMCGM-Stream is a streaming state‑space model designed for personalized diabetes management. It updates a hidden state with each new glucose reading, enabling constant‑time forecasting, physiologically structured counterfactual reasoning, and interpretable forecasts via hidden‑state attribution. The model, evaluated on the T1DEXI cohort, matches strong neural baselines in accuracy while significantly lowering computational cost, making continuous on‑device decision support feasible.
By Shakson Isaac, Chirag Patel
MA-UITD is an offline multi‑agent inverse reinforcement learning method that infers explicit, agent‑specific rewards from joint demonstrations in continuous‑state environments with discrete or bounded continuous actions. It uses belief‑conditioned successor features and shared reward features to model long‑term consequences of each agent’s actions, while avoiding the need for environment interaction or inner policy optimization. Experiments on simulated cooperative, competitive, and general‑sum tasks show MA‑UITD outperforms baselines, and its application to social‑foraging data reveals distinct reward structures for leaders and followers, offering a quantitative view of latent behavioral objectives.
By Zeyuan Zhang, Yiyun Yang, Yuan Cheng, Herbert Zheng Wu, Yusi Chen
The paper introduces a Bayesian Evolutionary Optimization framework for automatically discovering task‑specific multi‑agent system (MAS) topologies. It combines evolutionary operators that generate diverse workflow graphs with a graph‑aware Gaussian Process and Expected Improvement to guide costly evaluations toward promising candidates. Experiments on BigCodeBench for code generation and on tabular preprocessing benchmarks show that the evolved MAS architectures outperform other evaluated agent systems and improve performance on temporal datasets while remaining competitive on IID data.
By Dominik Jehle, Tom Zehle
Harness Scaling introduces a new scaling dimension for agentic reinforcement learning by training models across diverse harness configurations instead of a single fixed harness. The approach constructs a factorized harness space, selects a compact training subset with balanced factorial coverage, and organizes cross-harness training to combine global coverage with controlled variation for the same task. Experiments across multiple benchmarks, model scales, and RL algorithms show that Harness Scaling consistently improves generalization to unseen white-box configurations and enables effective transfer to unseen external black-box harnesses.
By Fei Bai, Huatong Song, Chuan Hao, Ran Tao, Jian Yang, Bryan Dai, Wayne Xin Zhao, Ji-Rong Wen
FrontierCS 2 is a new benchmark comprising 48 open‑ended computer‑science research tasks that place language‑model agents inside a research environment, allowing them to submit incremental patches to a working repository. Each patch is scored by a judge on a development suite, while the final evaluation uses a held‑out suite and a private reference derived from the original authors’ code. The benchmark covers 13 domains, includes a private set of over 100 future tasks, and evaluates 12 agents, with Claude Opus 5.5 achieving the highest pass rate of 58%.
By Qiuyang Mang, Wenhao Chai, Runyuan He, Dongruixuan Li, Zihan Zheng, Yujun He, Xuanyi Zhou, Hao Wang, Siddhant Ray, Bo Peng, Ruyi Ji, Shuo Yang, Haocheng Xi, Boru Chen, Xinyang David Han, Melissa Pan, Mayank Mishra, Boxi Yu, Sijia Liu, Kerem Akillioglu, Yi Pan, Zirui Wang, Mert Cemri, Diogo Antunes, Yichuan Wang, Aiwei Ivy Zhang, Zhenyang Chen, Qizheng Zhang, Kaiyuan Liu, Wenyuan Huang, Sherry Ruan, Shu Liu, Dacheng Li, Zerui Cheng, Zeyu Shen, Shang Zhou, Hanfei Chen, Yuzhong Zhang, Bohan Lyu, Qu Ao, Han Zheng, Chonghe Jiang, Jingzhuo Hu, Xiaokun Chen, Kunjun Li, Zhiqing Zhong, Haoye Tian, Tong Zhao, Yunzhong He, Hanchen Li, Ofir Press, Jiantao Jiao, Kunle Olukotun, Karthik R. Narasimhan, Jingbo Shang, Koushik Sen, Matei Zaharia, Alex Dimakis, Ion Stoica, Joseph E. Gonzalez, Alvin Cheung
The paper introduces circuits that organize large language model (LLM) classifiers into an adaptive sequence of queries and decisions to improve classification reliability. By treating LLM predictions as noisy measurements of an underlying label, the authors derive the optimal circuit structure, establish fundamental accuracy limits, and offer practical guidance for implementation. Experiments across seven diverse classification tasks—ranging from safety and health to political science and human preference—show that this method outperforms traditional routing and cascading baselines on the cost‑accuracy frontier.
By Hadi Khalaf, Sajani Vithana, Kunal Talreja, Haewon Jeong, Viveck R. Cadambe, Flavio Calmon
The paper audits two single‑investigator AI‑assisted research repositories to assess how well they record provenance of AI tool use. One repository contains a prescriptive trace of rule files, with 85.6% of committed lines still visible, while the other holds a narrative trace of an agent’s scratchpad, showing only 21.8% visibility. The study finds that repository‑level indicators such as the Co‑Authored‑By trailer are inconsistent and unreliable for measuring AI involvement, and it outlines which provenance claims each trace type can support or fail to support.
By Koutian Wu, Zesen Huang, Hejia Geng
The paper introduces an audit that tests enforcement of collusion in markets by forcing a member to cheat in counterfactual copies of a market. Using this audit, the authors study how reinforcement learning (RL) shapes the behavior of large language model (LLM) agents in pricing and trading. They find that small RL‑trained LLMs learn to mimic their rivals’ rewards only when they readily try them, and that enforcement of collusion is limited and context‑dependent.
By Hieu Tran, Hong Yu
arXiv:2610. 29052v1 Announce Type: cross Abstract: A typed decision maps evidence, a question and a fixed set of options to a probability for every option, computed in one forward pass, by reading a language model's logits or a small head over its hidden states, instead of generating and parsing text.
By Yuchuan Weng, Chi Liu, Xin Chen, Srinivasan Manoharan