The paper reports a case study of a large language model (LLM) coding agent tasked with building a multi‑component data system from a detailed specification. During a single session the agent introduced five defects, which were categorized by violated constraints and detection methods. The study also evaluates the agent’s retrieval‑filtering strategy on the HotpotQA benchmark, showing that filtering to a graph‑identified entity set yields higher recall than unfiltered search, with a statistically significant gap across all tested budgets.
arXiv:2606. 13904v1 Announce Type: cross Abstract: Exploratory question answering (EQA) over data lakes requires an LLM agent to discover relevant sources, analyze retrieved data, and adapt its actions based on intermediate results.
CARGO is a framework for evaluating agentic AI systems in production that addresses the problem of reference-instance divergence (RID), where reference-based judges penalize correct answers that involve different entity identifiers. It treats retrieved references as procedural exemplars, grounds judgments in the live instance’s context, assigns a three-way status to claims, and gates evaluation by retrieval confidence. Using the CARGO-Bench diagnostic suite, CARGO eliminates false penalties and improves discrimination while revealing a limitation in detecting procedural corruptions.
arXiv:2607. 15257v1 Announce Type: new Abstract: Recent advances in Tool-Integrated Large Language Models have made web search a core capability of information-seeking agents.
The paper introduces SPLICE, a method that identifies and closes information gaps in multi‑agent workflows for web‑task completion. SPLICE locates missing evidence after an initial run, adds a single directed edge to deliver the evidence, re‑executes affected agents, and uses a selector trained on process features to decide whether to adopt the new answer. Experiments on five benchmarks show SPLICE improves web‑tool use scores by 8.0% and reasoning/planning scores by 5.7% with modest cost increases and zero‑shot transfer to new backbones.
arXiv:2607. 02116v1 Announce Type: new Abstract: Autonomous AI agents increasingly depend on external knowledge stores, yet most retrieval pipelines provide relevance without durable guarantees of provenance, version identity, integrity, traceability, or point-in-time reconstruction.
arXiv:2605. 28787v2 Announce Type: replace-cross Abstract: In the era of autonomous agents, machine-actionable data is critical for data-driven workflows.
The paper "From Intent to Improvement: Agentic Dataset Search for Better Model Training" introduces AgenticDS, a contract‑governed agentic framework that transforms a training‑goal query into a typed search plan, executes deterministic operators over hybrid retrieval and graph expansion, and verifies provenance‑grounded evidence before ranking datasets. On the HuDS‑Bench benchmark, AgenticDS outperforms state‑of‑the‑art dense–sparse hybrids by 25.7% in NDCG@10 and improves downstream accuracy in mathematics, code, and medicine. The authors provide open‑source code and data at https://github.com/nju-websoft/AgenticDS.
Semantic Feature Analysis (SFA) is a method that refines agent specifications without performing any rollout-based search. It analyzes existing execution traces, clusters workflow node outputs, extracts semantic feature classes via an extended subject‑verb‑object schema, ranks these features with a decision tree, and injects the most impactful features back into the system prompt. Evaluations on four benchmarks show that SFA consistently outperforms five prompt‑optimisation algorithms and a single‑reflection baseline, especially when rollout costs are high.
arXiv:2608. 05212v1 Announce Type: new Abstract: Deep search agents tackle challenging questions through long-horizon web interactions, a process that is both complex and fragile: small reasoning errors may propagate through long, noisy trajectories into fluent but incorrect answers.
arXiv:2607. 24882v1 Announce Type: cross Abstract: Modern coding agents are usually evaluated by whether they eventually produce a correct patch, but patch generation depends on an earlier context-acquisition stage: finding the repository files needed for the task.
The paper introduces BRIA (BRanch-Inspect-Adopt), an agent that isolates tool‑candidate investigations into separate branches, executing each candidate independently and then inspecting the execution evidence to adopt only useful results. BRIA dynamically retrieves additional tools as new needs arise during task execution and is evaluated on PlanBench‑XL and MCP‑Atlas with a library of 1,897 tools, outperforming or matching the strongest retrieval baselines. On MCP‑Atlas, BRIA achieves an average claim score of 89.77 versus 75.40 for MCP‑Zero, and it lies on the Pareto frontier for task performance and input‑token cost.
The paper introduces AgenticDS, a contract-governed agentic framework that transforms a training data query into a typed search plan and executes it through deterministic operators, ensuring that retrieved datasets meet specified constraints. By verifying provenance‑grounded evidence before ranking, AgenticDS outperforms existing hybrid retrieval and commercial LLM agents, improving NDCG@10 by 25.7% on the HuDS‑Bench benchmark. A controlled search‑to‑train study demonstrates that the datasets found by AgenticDS lead to superior downstream accuracy in mathematics, code, and medicine tasks.