arXiv:2605. 27882v2 Announce Type: replace-cross Abstract: LLM-based agents score well on search benchmarks, yet real users consistently find results unsatisfying, revealing a persistent evaluation-experience gap.
arXiv:2606. 12871v1 Announce Type: new Abstract: Search Agents (SAs) typically leverage large language models (LLMs) to support complex information-seeking tasks by autonomously exploring web sources and synthesizing information into comprehensive responses.
xDailyBench is a new benchmark comprising 248 tasks across 51 real‑life scenarios, designed to evaluate large language models on everyday professional consultation. The tasks are based on actual user requests and assessed with detailed binary rubrics that capture both explicit instructions and implicit needs inferred from context. In tests of 11 leading models, the best achieved a 75.6% task‑level score, yet all models struggled more with implicit requirements, showing gaps of at least 9 percentage points.
WorldBench is a new multilingual benchmark that tests large language model agents on culturally grounded everyday workflows, offering 1,600 tasks in seven languages and eight cultures. The benchmark evaluates agents through structured sandbox actions and introduces Constrained Task Success (CTS), a metric that assesses task completion, minimal modification, and other complementary aspects via deterministic and LLM-as-a-Judge evaluations. Experiments show that even leading models achieve only 49.2% CTS, revealing significant gaps in correctness and state preservation across languages and cultures.
arXiv:2603.27476v3 Announce Type: replace Abstract: AI-powered people search platforms are increasingly deployed for recruiting, sales prospecting, and professional networking, yet no standardized be...
arXiv:2607. 24850v2 Announce Type: replace-cross Abstract: Recent advances in large language models (LLMs) have enabled search agents to autonomously tackle complex tasks across extended search and reasoning horizons.
arXiv:2608. 07700v1 Announce Type: new Abstract: Translating a natural-language question into a SPARQL query that can be executed against a large knowledge graph requires resolving lexical ambiguity, grounding surface terms in the target ontology, and producing graph patterns that are both syntactically valid and semantically faithful.
Despite the growing use of LLM-powered agents to solve multi-step tasks in complex environments, existing benchmarks rarely test state preservation, performance across languages, and application to re...
Large language model (LLM) agents are increasingly deployed as personal assistants. Existing evaluations, however, mostly use short, self-contained requests in static environments.
arXiv:2608. 10875v1 Announce Type: cross Abstract: Large language model (LLM) agents are increasingly deployed as personal assistants.
arXiv:2607. 15257v1 Announce Type: new Abstract: Recent advances in Tool-Integrated Large Language Models have made web search a core capability of information-seeking agents.
arXiv:2607. 21324v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) systems increasingly employ multiple LLM agents.
arXiv:2606. 15367v1 Announce Type: new Abstract: Deep research agents aim to solve complex knowledge-intensive tasks through long-horizon planning, evidence gathering, reasoning, and report generation.