arXiv:2608.27831v1 Announce Type: cross Abstract: Coding agents are now commonly evaluated on the SWE-bench family of benchmarks, whose tasks are built from curated GitHub issues--long, structured, a...
arXiv:2610.19659v1 Announce Type: new Abstract: People delegate multi-step work to LLM agents and read only the reply that the agent hands back. Current evaluation scores task completion and the fait...
arXiv:2609.23490v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly execute multi-step workflows through tool use and interaction with users and environments. However, curr...
The paper evaluates how robust large language models (LLMs) are when using retrieval‑augmented generation (RAG) in practical settings. It investigates whether RAG always outperforms non‑RAG approaches, whether adding more retrieved documents helps, and whether the order of documents matters, using a benchmark of 1,891 samples across five datasets and three task categories. Experiments with 11 LLMs show generally high retrieval robustness, but performance varies by task and prompting strategy, indicating that adopting RAG should be considered on a case‑by‑case basis.
arXiv:2609.15079v1 Announce Type: cross Abstract: Multi-agent LLM architectures, such as LangChain and AutoGen, largely assume English as the lingua franca for internal inter-agent communication, eve...
The paper introduces TRACE, a fine‑tuning framework for Retrieval‑Augmented Generation (RAG) that addresses conflicts between retrieved knowledge and a model’s internal knowledge. TRACE uses multi‑agent debate traces to identify correct and incorrect candidates and answer‑shift patterns, providing fine‑grained supervision for reliable knowledge‑source selection. It also incorporates an answer‑completeness regularization mechanism to prevent empty, overly short, or prematurely terminated responses, thereby improving robustness against misleading retrieved content and enhancing answer quality.
The paper investigates why large language models (LLMs) miss evidence in multi‑hop retrieval‑augmented generation (RAG). It introduces ExecutionRAG, a training‑free register that tracks open sub‑questions and their supporting evidence, allowing the authors to distinguish between true retrieval failures and search drift. Using this method on 1,000 MuSiQue questions, the study shows a significant reduction in missed evidence but also reveals that other error types, such as premature resolutions and abstentions, increase, shifting the bottleneck from search guidance to evidence verification.
arXiv:2610.14428v1 Announce Type: new Abstract: Enterprise workflow automation typically requires retrieving a complete and accurate set of relevant information from large operational document corpor...
arXiv:2609.18672v1 Announce Type: new Abstract: An AI assistant that calls tools makes two decisions on every request: which tool to invoke, and whether any available tool applies. In the usual desig...
arXiv:2610.27205v1 Announce Type: cross Abstract: Web agents are typically evaluated on English-language interfaces, leaving their ability to operate across the world's multilingual websites largely...
The paper evaluates how modern large language models use internal web search to answer factual questions. Using 783 static queries and 288 dynamic queries, the authors find that enabling retrieval improves accuracy on static questions but hurts confidence calibration. On dynamic queries, models often retrieve but still achieve less than 70% accuracy, mainly due to poor query formulation and source selection, indicating that internal web search works better as a quick verification tool than a full information‑retrieval system.
The study examines how three context‑engineering techniques—semantic layers, orchestration, and compaction—affect data agents that answer questions over heterogeneous data. Using seven large language models on 1,514 long‑horizon tasks, the authors identify three agent behavior groups (investigators, premature answerers, early stoppers) and show that each technique benefits only the group it aligns with, while sometimes harming others. Combining the techniques yields smaller net gains than the sum of individual effects, leading to guidance on selecting techniques based on agent behavior, task demands, and accuracy‑cost trade‑offs.