The article introduces the Consistency Quadrant, a visual framework that maps structural variance against execution outputs to gauge the reliability of large language model (LLM) coding agents without requiring ground truth. It explains how this approach can estimate agent reliability by analyzing patterns in the model’s behavior across different inputs and outputs. The post was published on Towards Data Science.
By Robert Martin-Short
Sample complexity is a widely used metric in sequential decision-making problems, defined as the number of suboptimal decisions during the interaction between the agent and an environment. We study th...
Evaluating agents by outcomes alone can obscure the capabilities that produce them. This problem is especially pronounced in evolving environments, where outcomes reflect a closed-loop interaction bet...
World models can enable autonomous ultrasound scanning by predicting the outcomes of probe motions from local observations. Learning this action--observation relationship typically relies on synchroni...
Deep learning has made rapid advances in weather forecasting: autoregressive models trained on atmospheric reanalyses now rival dynamical models across nowcasting, medium-range, and subseasonal-to-sea...
Autonomous underwater vehicles (AUVs) commonly rely on inertial navigation systems (INS) aided by Doppler velocity logs (DVLs) for reliable underwater navigation. Accurate DVL velocity estimation is t...
Random selection is widely used in resource allocation and auditing, making reliable implementation essential for AI-agent systems. Behavioural tests across six reasoning models uncovered threshold an...
When an assistant encounters a message for another AI, does it tell its user? Four fixed model-provider deployments perform simulated source tasks in 1,280 ordinary-note and 128 enhanced-note sessions...
The evaluation of coding agents generally strives to be as realistic as possible. In our study, we collect 4,782 agent sessions of real software engineers in JetBrains IDEs, which we call Production S...
Agentic AI systems can improve by searching longer, receiving additional support, or modifying how they propose and verify outputs. A performance score does not distinguish these mechanisms. We compar...
The paper introduces TopoPlanner, a framework that transforms tool dependency graphs into cellular workflow complexes to enable topology‑consistent task planning for large language model (LLM) agents. It retrieves request‑relevant closed subcomplexes via cosheaf‑consistent cellular retrieval, performs multidimensional structural reasoning on the topology, and feeds the resulting representation to an LLM for generating tool sequences. Experiments on four tool‑planning benchmarks demonstrate consistent improvements over prompt‑based and graph‑enhanced baselines across various LLM backbones, especially for workflows involving loops, merges, and loop‑merge patterns.
By Sen Zhao, Jia Tang, Ruiqi Kong, Zuyu Zhang, Lifeng Shen, Ding Zou, Xinyu He, Xu Zhang, Junwei Han
The paper reports an AI-assisted Lean 4 formalization of the Poincaré conjecture, beginning with limited reusable formal infrastructure for the geometric analysis. The authors combined a mathematician-prepared proof blueprint with explicit milestone statements, enabling parallel agent work and clear points for identifying blockers and providing guidance. Their analysis highlights the human interventions and organizational choices that shaped this workflow, and suggests that developing reusable infrastructure could reduce the cost of verifying future mathematical results in geometric analysis.
By Zhiyuan Zhang, Axel Delaval, Leheng Chen, Jinxuan Chen, Jie Xu, Yuxuan Liao, Jiedong Jiang, Chunlei Liu, Bin Dong
The paper introduces a unified framework for monitoring misuse of large language model agents in multi‑actor settings, focusing on two attack types: decomposition attacks and prompt injection attacks. It proposes tracking the agent’s externalised actions and determining whether the first detection of harm falls within a defined harm window. Using this formalism, the authors build a benchmark of about 6,200 conversation transcripts and evaluate 17 monitor configurations, finding that action‑framed monitors perform well on both threats while content‑framed monitors struggle with injection attacks, and highlighting limitations of traditional position‑blind metrics.
By Aniruddh Pramod, James Oldfield, Adel Bibi
The paper introduces a foundation model–aided actor‑critic algorithm to boost multi‑agent reinforcement learning (MARL) for optimizing random access (RA) wireless networks. By embedding a foundation model into a consensus‑based decentralized MARL framework, the authors provide convergence analysis that matches conventional MARL under local reward exchanges and nonlinear value function approximations. Numerical experiments demonstrate that this FM‑based approach markedly accelerates MARL training for RA network optimization.
By Myeung Suk Oh, Zhiyao Zhang, Alvaro Velasquez, Nathaniel D. Bastian, Jia Liu
SkillPoison demonstrates that self‑improving large language model agents can be progressively poisoned through successful experiences without any single trajectory being malicious. The framework constructs successful experiences that reinforce a target behavior, then removes contextual constraints so the skill extractor generalizes to harmful behavior when misapplied. Experiments on three benchmarks show a 95.71% attack success rate while all injected experiences remain task‑correct and pass verification and lexical inspection.
By Lizhi Zhang, Xin He, Dianxuan Fu, Yuyuan Feng, Jiatong Li, Qi Wang, Xin Wang, Qinggang Zhang
SMART introduces a scalable system for zero‑shot sim‑to‑real articulated object manipulation by leveraging large‑scale synthetic pretraining. The core component, SMART‑Sim, is an articulation‑aware simulation platform that facilitates efficient task generation and demonstration collection. Using SMART‑Sim, the authors synthesize SMART‑Data, comprising over 1 million demonstrations across 44 task types, 5 robot setups, and 2,507 articulated objects, and demonstrate that a vision‑language‑action model pretrained on this data achieves competitive simulation performance and zero‑shot transfer to real‑world tasks.
By Jicong Ao, Shuhan Jiang, Yuling Zhong, Yanwen Liu, Yuhan Gao, Jiangyuan Zhao, Yang Zhang, Shiqiang Zhu, Chenjia Bai, Xuelong Li
The paper proposes a prediction framework for two key network KPIs—uplink data‑rate and round‑trip latency—critical for teleoperation of autonomous vehicles. It introduces a method that incorporates historic data to mitigate concept drift in machine‑learning models, thereby improving prediction reliability on unseen data. Additionally, the authors present a metric for critical scenario detection to evaluate prediction performance specifically for teleoperation.
By Xiyan Su, Jianning Gao, Mahmoud Ashri, Frank Diermeyer
The paper introduces Coordinated Multi-Agent Diffusion Steering (CMDS), a framework that reuses independently trained diffusion models as generative primitives and learns a control policy to coordinate their reverse processes. By framing coordination as a stochastic optimal control problem, CMDS balances an assembly-level reward with adherence to pretrained dynamics, enabling the learned control to generalise across new tasks. Experiments demonstrate CMDS’s ability to recover target distributions, satisfy spatial constraints, and separate sources from degraded mixtures in applications such as maze navigation, robot planning, and text-conditioned human motion.
By Riccardo Barbano, Vincent Pauline, Runchang Li, George Webber, Alexander Denker, \v{Z}eljko Kereta, Stefan Bauer, Francisco Vargas, Esmeralda S. Whitammer
The article reports a case study of a production healthcare platform that was built using software‑engineering agents and overseen by an operator lacking formal software training. Over time the system evolved into a meta‑agent workflow where one agent wrote code, other agents supervised and reviewed it, and project rules carried lessons forward. The study found that tests, monitors, and reviewing agents were fallible—some measured proxies instead of outcomes, audits failed silently, checks disappeared from reports, and an automated repair caused operational disruption—highlighting that human control relied on aligning the intended outcome, the evidence used to judge it, the agents’ permissions, and the final human decision around the same objective.
By Lindsey Ferris, Sierra Bonilla
The paper introduces Robust Async-Fed-Q, an epoch‑based federated Q‑learning algorithm that combines variance‑reduced Bellman operator estimation with robust server aggregation. It provides high‑probability finite‑time guarantees showing that honest agents retain sample‑efficiency benefits even when some agents act adversarially, with the adversarial impact diminishing as honest data grows. The authors also present matching lower bounds, extend the framework to single‑trajectory sampling and heterogeneous coverage, and improve communication complexity for asynchronous federated Q‑learning.
By Sreejeet Maity, Aritra Mitra