The paper introduces an Item Response Theory (IRT)–based indicator that identifies likely mislabeled items in large language model (LLM) benchmarks with 95% precision among the top 200 examples across seven preference and multiple-choice datasets, using responses from 114 models. It outperforms a supervised classifier and attributes the mislabels to mechanical labeling heuristics, inherited annotation errors, and inherently ambiguous items. The IRT analysis also reveals that reward models tend to specialize in stylistic preference rather than factual knowledge, and pinpoints a frontier reward model that aligns with detected mislabels at 78% accuracy compared to 38% for other models, suggesting benchmark contamination or over‑optimization.
By Sander Land, Daniel M. Bikel
The paper proposes treating the choice of monomial ordering for Gröbner basis computations as a reinforcement learning problem, using domain-informed reward signals that reflect actual computational cost. By training policies over the space of admissible orderings, the authors demonstrate that the learned strategies outperform traditional static heuristics such as GrevLex on benchmark problems from systems biology and computer vision. The resulting policies also resist simplification into interpretable models, suggesting that deep reinforcement learning captures complex geometric structure beyond conventional approaches.
By R. Caleb Bunch, Alperen A. Erg\"ur, Melika Golestani, Jessie Tong, Malia Walewski, Yunus E. Zeytuncu
arXiv:2605.12340v5 Announce Type: replace-cross
Abstract: Learning-to-Defer (L2D) methods route each query either to a predictive model or to external experts. Real-world deployments require handling...
By Dang Hoang Duy, Yannis Montreuil, Maxime Meyer, Axel Carlier, Lai Xing Ng, Wei Tsang Ooi
The paper introduces Generalized Gibbs Ensemble Weighting (GGEW), a probabilistic framework that assigns weights to forecasting models using a Gibbs-style exponential transformation of normalized predictive loss. GGEW extends basic weighting through numerical stabilization, diversity-aware score corrections, and online hyperparameter adaptation, yielding variants such as Stable Gibbs weighting, Directional Gibbs-NCL, and Symmetric Gibbs-NCL. The authors evaluate GGEW on M4 competition submissions and real-world datasets (Monash Traffic, Electricity, Solar), finding that Gibbs-style adaptive weighting is competitive across various settings, though performance varies by dataset, horizon, and deployment protocol.
By Prasen R. Nuthanakaluva, Nava K. Gaddam
The paper introduces a method for tackling combinatorial optimization (CO) problems—often NP‑hard—by leveraging GFlowNets to sample solutions from the solution space. It designs Markov decision processes tailored to various CO tasks and trains conditional GFlowNets, incorporating efficient training techniques for long‑range credit assignment. Experiments on synthetic and realistic datasets show that these GFlowNet policies can efficiently locate high‑quality solutions, and the implementation is publicly available.
By Dinghuai Zhang, Hanjun Dai, Esmeralda S. Whitammer, Aaron Courville, Yoshua Bengio, Ling Pan
HARTS (Hybrid‑Attention RL over Tree Structures) is a new system that jointly plans microbatches, data‑parallel replica assignments, and microbatch‑slot schedules to efficiently train agentic reinforcement learning models with hybrid attention over arbitrary rollout trees. It uses prefix compression and a linear‑time algorithm for chunkwise linear attention to avoid recomputing shared prefixes, enabling activation recomputation and bounded state replay while preserving trajectory‑wise training benefits. In experiments on an Agentic RL workload derived from SWE‑bench tasks, HARTS delivers 4.81–4.87× speedups in forward, backward, and gradient computations across multiple parallel configurations, with numerical differences comparable to baseline self‑rerun variation and a similar reward trend over the first 120 training steps.
By Boyuan Meng (Ant Group, China), Peihua Bao (Ant Group, China), Hong Liu (Ant Group, China), Xiaowei Zhu (Ant Group, China), Chao Wang (Ant Group, China), Gen Li (Ant Group, China), Zhenxuan Pan (Ant Group, China)
Large reasoning models generate chain-of-thought (CoT) before answering but struggle with safety alignment and can be misled by flawed premises. The paper introduces RECAP, a reinforcement learning approach that trains models to override flawed reasoning paths and produce safe, helpful responses without extra training cost. RECAP improves safety, jailbreak robustness, and reduces overrefusal while preserving core reasoning abilities and inference token budget.
By ShengYun Peng, Pin-Yu Chen, Eric Smith, Song Jiang, Hongyuan Zhan, Haozhu Wang, Mahesh Pasupuleti, Duen Horng Chau, Jianfeng Chi
arXiv:2604.06079v2 Announce Type: replace
Abstract: Graphics Program Synthesis is pivotal for interpreting and editing visual data, effectively facilitating the reverse-engineering of static visuals...
By Juekai Lin, Yun Zhu, Honglin Lin, Sijing Li, Tianwei Lin, Zheng Liu, Xiaoyang Wang, Wenqiao Zhang, Lijun Wu
arXiv:2608.28046v1 Announce Type: cross
Abstract: Collective behaviour in living systems is usually modelled as the outcome of a \emph{direct} social drive: agents are rewarded, or hard-wired, to ali...
By Gorka Mu\~noz-Gil, Andrea L\'opez-Incera, Vide Ramsten, Giovanni Volpe, Thomas M\"uller, Hans J. Briegel
Agentic‑Kube is a cooperative multi‑agent reinforcement learning framework for Kubernetes pod placement that splits the multi‑objective scheduling problem into cost minimisation, anti‑affinity fault tolerance, and vector resource balancing, each handled by a dedicated sub‑agent. It uses a bipartite Graph Convolutional Network to model host‑pod dependencies, a two‑stage monotonic QMIX value factorisation network for joint action coherence, and a plurality voting consensus with action feasibility masking. Evaluations on Google Kubernetes Engine and large‑scale clusters show Pareto‑efficient placements, a 53% reduction in anti‑affinity collisions, a 65% spot instance allocation ratio, and sub‑30 ms decision latencies up to 1,000 nodes without container restarts.
By Hamed Hamzeh
The paper introduces Synthetic Linguistic Agency (SLA), a framework that defines linguistic agency in terms of embodiment, participation, and precariousness. It presents two studies: one that operationalizes SLA criteria and identifies existing systems, and another that builds an Embodied Mortal Agent (EMA) using mortality‑grounded reinforcement learning. Experiments show the EMA’s linguistic choices depend on its body and social history, influence partner behavior, and adapt over time, demonstrating SLA in an artificial agent.
By Sixin Chen, Taizhou Chen
The paper introduces EXPO, an algorithm that automatically optimizes the meta-prompt—specifically the task description and meta-instruction—for large language model agents in sequential decision-making tasks such as Bayesian optimization and multi-armed bandits. Building on adversarial bandit techniques to handle non-stationary rewards, the authors extend EXPO to EXPO-ES, which also optimizes exemplars (historical interactions) within the meta-prompt. Experiments demonstrate that these methods significantly improve the performance of LLM-based agents in sequential decision-making scenarios.
By Mingze Kong, Zhiyong Wang, Yao Shu, Zhongxiang Dai
The paper introduces VICT, a method that leverages the internal structure of verifiable tasks to perform fine‑grained credit assignment for long‑horizon LLM agents. VICT exposes executable or evidence‑backed atoms from a task’s terminal verifier and traces them back to actions via dependency‑valid proof edges, redistributing advantage only along these edges. This approach improves performance on ALFWorld and WebShop compared to outcome‑only training and matches recent fine‑grained credit methods without requiring additional critics, labels, or inference‑time verifier access.
By Pengcheng Li, Zhengyang Zhang, Dongxu Zhang, Sui Huang, Shaohua Ma
The paper introduces prequential posteriors, a Bayesian approach that uses a predictive‑sequential loss function to update deep generative forecasting models (DGFMs) when new data arrive. By adopting a consistency notion suitable for model misspecification, the authors prove that both the loss minimizer and the posterior concentrate on parameters with optimal predictive performance. Scalable inference is achieved with parallelisable waste‑free sequential Monte Carlo samplers that employ preconditioned gradient kernels, and the method is validated on synthetic and real meteorological time‑series data.
By Shreya Sinha-Roy, Richard G. Everitt, Christian P. Robert, Ritabrata Dutta
CommerceVibe is a system that generates e‑commerce creatives by synthesizing executable HTML/CSS code conditioned on product images, design requirements, and product information. It uses dual‑feedback reinforcement learning, combining rule‑based checks for text readability, product visibility, and layout validity with visual feedback from a vision‑language model that evaluates perceptual and commercial aspects. After fine‑tuning a large language model on 28,000 examples and applying dual‑feedback reinforcement learning, CommerceVibe achieves a weighted score of 94.0/100 on a 1,300‑case benchmark, outperforming both its SFT‑only counterpart and external models, and is validated by expert blind evaluations.
By Yajiao Xu, Jin Zhang, Jiangbo Ai, Tao Jiang, Mo Xu, Lina Huang, Chengfu Huo
The paper challenges the common assumption that the successor measure in reinforcement learning is approximately low-rank, showing instead that a low-rank structure emerges in a shifted successor measure that ignores initial transitions. It provides finite-sample guarantees for estimating this low-rank approximation, introduces Type II Poincaré inequalities to bound spectral recoverability, and links the necessary shift to the decay of high-order singular values and local mixing properties. Experiments confirm that shifting the successor measure improves goal-conditioned RL performance.
By Bastien Dubail, Stefan Stojanovic, Alexandre Prouti\`ere
The article argues that Monte Carlo Tree Search (MCTS) and every‑visit Monte Carlo (MC) control are essentially the same method, differing only in terminology and presentation. It shows that MCTS’s four stages—selection, expansion, simulation, and backup—can be reduced to two core operations: sampling trajectories under the current policy and performing every‑visit MC updates. The note aims to make this equivalence explicit and easier to recognize.
By Xianyi Wu
The paper introduces Replicant, a deep reinforcement learning framework that learns to evade malware detectors under a strict label‑only black‑box threat model. Replicant generates reusable policies for modifying malware samples and deciding when to query the target, and it transfers across different samples, detectors, and feature spaces. In experiments on seven Android malware detectors and three feature spaces, Replicant achieves a mean attack success rate of 78.8%, outperforming state‑of‑the‑art methods by 20.9%–39.2% and providing a stronger signal for adversarial training to harden detectors.
By Shae McFadden, Ilias Tsingenopoulos, Mario D'Onghia, Alexander Herzog, Myles Foley, Chris Hicks, Lorenzo Cavallaro, Fabio Pierazzi
arXiv:2606.05122v2 Announce Type: replace
Abstract: Large language models are increasingly evaluated by other models, raising a natural question: can a model predict how a judge will score its own ou...
By XiuYu Zhang, Yi Shan, Junfeng Fang, Zhenkai Liang
arXiv:2608.27757v1 Announce Type: new
Abstract: Searchless chess networks reach human master strength from a single forward pass by imitating a stronger teacher: the strongest, Leela Chess Zero's (Lc...
By Szymon Mi{\l}osz, Piotr Duch, Szymon Grabowski