Tool use, function calling, orchestration and the protocols that let models act rather than only answer.
arXiv:2611. 05586v1 Announce Type: new Abstract: As autonomous AI agents enter permissionless onchain markets, selecting new service providers remains a core challenge.
MA‑SafeDiffuser is a multi‑agent extension of SafeDiffuser that incorporates diffusion‑based trajectory planners with finite‑time diffusion invariance guarantees for joint‑agent safety specifications. It formalizes joint safe sets as intersections of per‑agent and pairwise barrier sets, and derives both centralized and decentralized constrained denoising procedures that maintain invariance under mild assumptions. The approach also tackles local‑trap and deadlock issues using time‑varying specifications and liveness CBFs, and introduces a lightweight benchmarking suite including a multi‑agent Maze2D domain, showing reduced violation counts compared to unconstrained diffusion baselines while preserving planning quality.
The paper introduces a challenge‑aware adaptive recruitment system for medical multi‑agent reasoning. An external CHALLENGER monitors the evolving reasoning trace to detect factual or expertise gaps, triggering targeted knowledge retrieval or specialist recruitment mid‑discussion. Experiments on the MEDAGENTSBENCH hard subset show that challenge‑aware in‑between retrieval and Challenger‑triggered specialist recruitment improve accuracy, while naive combinations degrade performance, underscoring the importance of timing and coordination in resource recruitment.
VirtualTrialHarness (VTHarness) is an agent framework designed to streamline the operationalization of virtual trial protocols by selecting reusable procedural skills for each workflow stage and using explicit contracts to ensure consistent study decisions. It introduces VTBenchmark for evaluating source selection and cohort construction, achieving 95.1% workflow success after requirement clarification versus 60.4% with a standard procedural corpus. The framework demonstrated improved source recovery across language models, increased overlap with analyst code in Mayo Clinic cases, and enabled reference‑matched cohort execution across multiple MIMIC‑IV tasks, supporting investigator review within the Mayo Clinic Virtual Trials Program.
The paper introduces Conformal Reservation for Orchestration (CRO), an online algorithm that uses past execution traces to allocate compute and bandwidth for agentic AI systems at dispatch time. CRO provides finite‑sample, distribution‑free coverage guarantees, ensuring that resource reservations meet specified probabilistic deadlines. Simulations demonstrate that CRO uniquely satisfies target coverage across all risk levels while reducing over‑reservation—and thus resource waste—by about half compared to a Chebyshev‑bound baseline.
Agent Harness Awareness addresses the gap between tool descriptions and real‑time operational suitability by having the host provide a verified, up‑to‑date state snapshot before an agent dispatches an action. In controlled benchmarks, this approach cut observable policy regret by nearly 97 % and boosted single‑dispatch completion from 69.4 % to 95.8 %. In live, fault‑injected tests, verified state eliminated excess operational cost and achieved 100 % completion across all model‑by‑fault scenarios.
AvalancheBench is a multimodal enterprise analytics benchmark that tests data agents’ ability to recover a latent world from observed data, using 58 questions across text, images, and documents. It evaluates agents on analytical fidelity, monetary cost, and a new capability‑adjusted token cost (CATC) metric, revealing that misinterpreting requirements and misuse of semantic operators are the biggest sources of error. The benchmark highlights how errors can propagate through dependent questions, affecting overall performance.
The paper introduces OPSD, a method that uses on‑policy self‑distillation to train a context‑free student model from a teacher conditioned on user memory, combined with a verify‑then‑regenerate router that directs queries to either weight or text memory. OPSD enables weight memory to compete with or surpass traditional text‑based memory systems on the HaluMem benchmark, reducing token usage per query. The study also explores how different training recipes influence weight memory performance and offers guidance for integrating it with retrieval in persistent, multi‑session agent deployments.
The paper investigates confirmation bias in Agent-as-a-Judge systems, which evaluate open-ended agent tasks by gathering evidence from the agent’s trajectory. It demonstrates that a single false pre‑screening note can overturn up to 40% of correct verdicts across multiple judges and datasets, even when the judge collects decisive evidence. The authors classify the resulting errors into over‑trust, hallucination, and under‑investigation, noting that over‑trust dominates in larger Claude judges, and show that lightweight mitigations reduce but do not eliminate the bias.
The paper investigates how vision‑language models (VLMs) perceive uncertainty in visualizations and make decisions based on visual data. Two studies were conducted: one on agents’ ability to quantify uncertainty in charts, and another on their decision‑making when presented with images. Results show GPT‑4o best perceives uncertainty in pie and bar charts, while agents align most closely with rational decision theory when uncertainty is shown via circle charts, icon arrays, or text.
The paper proposes a four‑layer architecture—Signals, Domains, Claims, and Dimensions—to preserve and revise organizational interpretations of evidence. It introduces a truth‑maintenance network that updates the standing of interpretations as new evidence arrives and uses validated, versioned Dimensions to position these interpretations along named axes. Experiments show that incorporating Dimensions improves retrieval performance, with a 0.028 nDCG@10 gain for the Dimension term and overall nDCG@10 increases of 0.081 and 0.053 over baseline configurations.
Reglos is a training‑free personal memory agent that continuously updates an append‑only record of user interactions, allowing newer preferences to override older ones and enabling selective forgetting of facts. Its memory harness evolves through recursive self‑improvement: a coding agent generates patches from failure traces, a critic from a second model family screens them, and a fixed evaluator decides which patches to accept. Using the same backbone across systems, Reglos surpasses state‑of‑the‑art memory agents such as Mem0, MemOS, and EverOS, achieving 61.3 % and 84.0 % on PersonaMem‑v2 and LongMemEval‑S with Qwen3.8‑27B, and 57.5 % and 87.2 % with GPT‑5.6 Luna.
EnzymeRSI is a dual‑loop recursive self‑improvement system that separates the evolution of harness assets (outer loop) from the development of prediction and selection algorithms (inner loop), closing the loop with a cloud laboratory for physical feedback. The system improved ProteinGym Spearman correlation by 0.044 across five assays, achieved high correlations on CD19 and OPSD, and accelerated ThermoMPNN and TadA‑Bench performance. In prospective wet‑lab tests, EnzymeRSI identified variants with markedly higher activity, including a cytidine deaminase variant with 90.2% 5mC conversion and a DNA polymerase KOD‑E variant with 31.1‑fold increased catalytic efficiency.
ElecLoad is a context‑aware agent that improves electricity load forecasting by adaptively organizing the forecasting process around multiple events. It analyzes each event’s relevance, timing, and interactions with historical load, then decides whether to forecast the entire horizon or partition it into segments. Sub‑agents generate initial forecasts for each segment, and the main agent refines them based on feedback, achieving state‑of‑the‑art performance on several datasets.
The paper introduces a shared framework that adapts Cosmos world foundation models for real‑time interactive world simulation in Physical AI. By streaming visual observations in response to actions and scene controls, the framework allows users and embodied agents to interact with evolving virtual environments, supporting both prediction of visual consequences and generation of observations from an external simulator’s physical state. The authors demonstrate the approach in camera navigation, robot manipulation, controllable video transfer, and multi‑view autonomous vehicle simulation, providing a common foundation for developing interactive virtual environments for Physical AI training and evaluation.
LifeVal is a new benchmark that evaluates AI model performance on personalized agentic consumer tasks, covering over 1.7 trillion US dollars in annual willingness-to-pay across 148 task categories such as managing finances and finding childcare. The benchmark was built using a survey of more than 3,800 consumers and cross‑referenced with national time‑use studies. Task creators, who met strict expertise thresholds, designed AI evaluation tasks personalized with real ChatGPT conversations, memories, reference files, and personal context, and all tasks underwent at least three rounds of human quality control before inclusion.
The paper introduces a method for efficient agent evaluation by using cached traces from task executions. It proposes summarizing each trace with an LLM and embedding the summary to capture task-relevant signals while reducing system-identifying noise. Using these representations, the authors build low-dimensional system embeddings and predict a new system’s benchmark score, achieving lower error than baseline models on SWE-bench Verified and Terminal-Bench 2.0 with only a single executed task.
arXiv:2611. 06868v1 Announce Type: new Abstract: In this work, we evaluate a simple inference-time scaling approach based on \textit{turn-level best-of-N sampling} (\method) for large language model (LLM) agents on complex, long-horizon tasks.
FrontierPhysics is a new end‑to‑end benchmark for AI agents in physics, comprising 56 tasks created by 42 physicists across nine domains of theoretical, computational, and experimental physics. Each task requires an agent to review literature, devise a research plan, conduct experiments, and produce structured outputs and manuscripts, with evaluation based on deterministic verifiers and expert rubrics. In tests of 22 models, none exceeded 46 of 168 rollouts, revealing that agents struggle mainly with rigorous reasoning and expert intuition rather than physics knowledge or execution.
LIBRA is a framework that transforms a flat catalog of tools into an executable library by having a large language model propose contracts for tool invocations, reusable outputs, state changes, and success conditions. These proposed compositions are then tested in a sandbox, and only the relations that succeed at runtime are retained, eliminating the need for dependency annotations or task‑specific training. The approach improves one‑shot recall and NDCG on several benchmarks, and further enhances retrieval performance after multiple iterations.