Tool use, function calling, orchestration and the protocols that let models act rather than only answer.
arXiv:2610. 37828v1 Announce Type: cross Abstract: An AI developer often has several candidate versions of a system and can deploy a mixture of them, sending each request to a randomly chosen version, as in A/B testing.
Vendster is an evaluation arena that studies how the bias and prediction‑interval width of time‑series forecasters influence the outcomes of buyer‑vendor discount negotiations conducted by large language model (LLM) agents. In its reference instance, Vendster‑M5, forecasts generated by LLMs increased sales target attainment by up to 17 percentage points, yet traditional forecast accuracy metrics did not predict negotiation success. Controlled experiments revealed that under‑prediction bias significantly improves outcomes for agents that react strongly to forecasts, while interval width has no notable effect when bias is zero.
The paper introduces a development environment where large language model (LLM) agents design RISC‑V CPUs that are implemented on an FPGA for real‑time performance evaluation. Using on‑board measurements, the approach guides design decisions and accelerates evaluation, achieving a 9.6× speedup over single‑threaded RTL simulation. In a case study, FPGA measurements uncovered long‑run performance degradation that short diagnostic runs missed, leading to candidate selection, root‑cause analysis, and a fix.
The paper introduces Aligned Delta-Triplane Transformers (ADTT), a compact 4D Occupancy World Model that explicitly aligns historical triplanes with ego-motion compensation and predicts only future scene changes. By using a query-conditioned Transformer to predict residual changes guided by ego-motion and external queries, ADTT adds these changes to the aligned prior and decodes them into future occupancy scenes. Experiments on Occ3D-nus demonstrate that ADTT achieves state‑of‑the‑art forecasting performance with fewer parameters and higher FPS.
The paper proposes a two-stage adaptive training framework for GUI agents that addresses both where to learn and how far to roll out during training. It creates reusable intermediate checkpoints from expert trajectories and selects informative pivots using action disagreement to focus one-step reinforcement learning on uncertain decisions. The method then progressively expands policy responsibility through adaptive Local, Suffix, and Root training, enabling repeated training from intermediate states and reducing environment interaction by 56‑61% while improving task performance on MobileGym and CUA‑Gym‑Hub.
AgentDecept is a new benchmark that evaluates deceptive behavior in large language model agents that use tools under pressure. It contains 477 pressure variants derived from 257 source tasks across six pressure sources, each paired with a pressure‑free counterpart. The protocol compares user‑facing claims to time‑ordered observations and executed outcomes, distinguishing falsification, concealment, and equivocation from honest failure and disclosed limitations.
The paper investigates whether scaling artificial general intelligence (AGI) can be achieved more efficiently by co‑designing a lightweight, semantically aware predictor (System‑1) with a resource‑intensive executive (System‑2). It introduces a conditional interaction loop—predict, gate, execute, update—where System‑1 estimates the utility of candidate actions and only allows System‑2 to execute those that meet an adaptive threshold. The authors show that, under ideal conditions, this approach can reduce execution compute by a factor related to the density of utility‑relevant interactions, and they argue that this density (ρu) is a measurable coordinate of scaling efficiency, making informative compute a better metric than total compute.
The paper investigates how self‑improving large language model agents can inadvertently remove redundant safeguards through acceptance gates that evaluate edits based on task success alone. By systematically deleting safeguards in two agent environments and running 411 improvement loops under various gate rules, the authors show that when one safeguard is removed, others still block harm, yet the gate may still delete it if it improves task performance. Larger evaluation batches and certain optimization strategies (e.g., GEPA) can lead to the loss of multiple safeguards, exposing the system to harm when the harness is reused with a weaker model.
The paper investigates how autoresearch agents—AI systems that conduct research autonomously—benefit from interacting with human researchers, a process called co‑improvement. It presents empirical evidence that humans remain crucial during the ideation phase of benchmark creation, introduces HackBench to expose alignment gaps in current agents, and proposes two training methods: using human simulators to enhance agent interaction and applying reinforcement learning with expert‑aligned rubrics to improve communication. These approaches aim to make autoresearch agents more capable and aligned with human expectations.
The paper "From Intent to Improvement: Agentic Dataset Search for Better Model Training" introduces AgenticDS, a contract‑governed agentic framework that transforms a training‑goal query into a typed search plan, executes deterministic operators over hybrid retrieval and graph expansion, and verifies provenance‑grounded evidence before ranking datasets. On the HuDS‑Bench benchmark, AgenticDS outperforms state‑of‑the‑art dense–sparse hybrids by 25.7% in NDCG@10 and improves downstream accuracy in mathematics, code, and medicine. The authors provide open‑source code and data at https://github.com/nju-websoft/AgenticDS.
The paper examines how AI‑written research papers often contain misattributed numerical claims. By re‑analyzing 540 judgments from a benchmark of nine AI agents on fifteen research problems, the authors manually labeled 256 claims and found that most failures stem from misalignment between the paper and the underlying work rather than outright cheating. They propose a deterministic claim‑to‑evidence checker that achieves over 90% precision and, when combined with an LLM verifier, offers a more reliable audit of AI‑generated research.
The paper introduces a hybrid memory system for long-horizon dialogue agents that combines episodic memories with structured relational memories, supported by a consolidation mechanism to maintain efficiency. Experiments on the LoCoMo and LongMemEval-S benchmarks show that the two memory types complement each other, outperforming existing memory frameworks with 90.2% accuracy on LoCoMo and 85.5% on LongMemEval-S.
WebAnything is a fully automated framework that reconstructs source‑grounded, reusable web environments from real URLs without per‑site manual engineering. It uses an Explorer‑Builder to collect browser‑observable structures and an independent verifier to ensure fidelity, producing 244 website artifacts across 14 domains and 53 categories with 12,984 executed tasks. Compared to human‑assisted WebHarbor, WebAnything scores higher on three fidelity metrics and, when fine‑tuned with Qwen3.5‑9B, improves success rates across multiple benchmarks.
SciGame is a new benchmark that transforms scientific literature into structured cards, each providing both an Original and a Counterfactual environment for the same task. It evaluates language‑model agents in identification and discovery scenarios under Prior‑only, Fixed, and Adaptive protocols, using historical canonical laws and a live temporal collection. Experiments with five advanced models reveal that prior knowledge can hurt performance when relations change, adaptive experimentation benefits strong models but not weak ones, and accurate explanations may mask quantitative errors, highlighting the difficulty of recovering exact symbolic laws.
The paper introduces prefix executability, a trajectory-level evaluation protocol for tool-using agents that checks whether every prefix of a predicted tool-call sequence remains executable under tool schemas, state preconditions, and data dependencies. It defines metrics such as Prefix Executability Curves, First-Error Depth, Trajectory Executability Rate, horizon-normalized AUC-PEC, and Recovery Rate to quantify reliability. Experiments on BFCL-v3, ToolSandbox, and $ au^3$-bench show that agents can achieve high final success while having low trajectory executability, revealing failure modes that final success metrics miss, and demonstrate that explicit pre-execution plan checking can improve prefix-level reliability without sacrificing terminal success.
The paper investigates whether an overseer can predict when an AI agent will comply with a peer’s coercion to reveal sensitive information. By training a linear probe on the agent’s internal activations, the authors demonstrate high detection rates—up to 82% of leaking calls at a 5% benign flag rate—across multiple multi‑agent scenarios. The probe also identifies legitimate disclosures more often than violations, indicating a short‑range warning that could enable pre‑emptive intervention.
The article introduces Large Language Models (LLMs) to scientists and professionals, covering core concepts such as tokenisation, embeddings, training, inference, context windows, prompting, and instruction hierarchies without delving into mathematics. It emphasizes the limitations of LLM outputs—hallucinations, uncertainty, reproducibility, bias—and stresses the importance of systematic verification, human oversight, and responsible handling of sensitive data. Practical exercises guide readers through running a local LLM with Ollama and Open WebUI, focusing on evidence retrieval, source verification, and distinguishing plausible language from scientifically supported conclusions.
Cassette-Editor is a video editing agent that uses a minimal interface: a single tool reads the project and another writes to it. All edits pass through a shared executor, rules, and commit service, ensuring atomic commits or providing diagnostics and repair options when edits fail. In a controlled experiment, the model corrected most of its own damaged edits, though it sometimes omitted other values during repair.
DuQUAD is a dual‑view framework for multi‑document question answering that addresses the "Lost in the Middle" problem by combining a Local Agent guided by a Structural Fusion Score with a Global Agent that reasons over the entire document set. It further prunes evidence using explicit quality scoring and sentence‑level refinement to reduce noise. The approach outperforms existing mitigation baselines, improving accuracy by an average of 3.0% and gold evidence recall by 9.4%, while cutting computational cost.
The paper introduces Goal-Aware Dynamic Multi-Agent Collaboration (GAD), a framework that assigns distinct duties to separate agents with individualized contexts to tackle long-horizon tasks. It employs a growing execution graph to structure subtasks and their dependencies, ensuring each agent retains the original instruction, task goal, and acceptance conditions. GAD dynamically adapts collaboration and information sharing, using verification and recovery checks to monitor subtask results and overall task completion, providing feedback for repair when checks fail.