arXiv:2603. 27476v2 Announce Type: replace Abstract: AI-powered people search platforms are increasingly used in recruiting, sales prospecting, and professional networking, yet no widely accepted benchmark exists for evaluating their performance.
Q2D-Web is a new large‑scale benchmark for agentic Retrieval‑Augmented Generation (RAG) systems, featuring a 190 million‑document web corpus and 70 k machine‑reformulated search queries in ten languages. It supplies three sets of relevance judgments—agent citations, production rankings, and a combined set enriched with LLM‑based labels—to evaluate first‑stage retrievers. Experiments on 13 retrievers show consistent ranking across judgment sets but significant variation across domains, languages, and query types, and demonstrate that a carefully sampled sub‑corpus can approximate full‑corpus evaluation with minimal loss in Recall@1000.
arXiv:2606. 12871v1 Announce Type: new Abstract: Search Agents (SAs) typically leverage large language models (LLMs) to support complex information-seeking tasks by autonomously exploring web sources and synthesizing information into comprehensive responses.
The study examines how conversational LLM agents—specifically ChatGPT, Claude, Grok, and DeepSeek—use Web search, combining real user interactions with controlled API experiments. It finds that agents differ in when they decide to search, how they craft queries, and which domains they favor, and that more frequent searching does not always improve answer quality. While most responses are grounded in search results, some claims are unsupported, raising attribution concerns.
The paper introduces SetwiseEvalKit, a benchmark with nine dimensions and 28K rubrics for evaluating document sets, highlighting that current rerankers only achieve up to 45% coverage and struggle with cross‑document coordination. It proposes Rubric4Setwise, a training‑free method that transforms rubric criteria into selection signals, outperforming other methods in downstream generation tasks across both short‑form and long‑form scenarios. The study demonstrates the importance of evaluating and optimizing document sets as a whole rather than independently.
arXiv:2606. 05308v1 Announce Type: new Abstract: With PRECISE, we extended Prediction-Powered Inference to produce bias-corrected estimates of ranking evaluation metrics by combining a small human-labeled set with a large LLM-judged set.
arXiv:2607. 03886v1 Announce Type: cross Abstract: Sponsored search plays a crucial role as a revenue stream for search engines, wherein advertisers competitively bid on keywords that align with the users' search queries.
arXiv:2605. 27882v2 Announce Type: replace-cross Abstract: LLM-based agents score well on search benchmarks, yet real users consistently find results unsatisfying, revealing a persistent evaluation-experience gap.
arXiv:2610.15409v1 Announce Type: new Abstract: Enterprise agents need retrieval evaluation sets that expose domain-specific failures and guide developers towards efficient deployment tradeoffs. Neit...
arXiv:2606. 00809v1 Announce Type: new Abstract: Many real-world conversational settings for knowledge discovery, including podcasts, hiring screens, and marketplaces, require a purpose-driven understanding of a person.
arXiv:2602. 23234v4 Announce Type: replace-cross Abstract: Large-scale commercial search systems optimize for relevance to drive successful sessions that help users find what they are looking for.
arXiv:2608. 02446v1 Announce Type: cross Abstract: Relevance evaluation plays a crucial role in personalized search systems, serving as a guardrail alongside user engagement metrics to ensure that search results align with user queries and intent.