arXiv AI

LakeQA: An Exploratory QA Benchmark over a Million-Scale Data Lake

arXiv:2606. 10460v1 Announce Type: cross Abstract: Recent large language models (LLMs) have shown rapid progress in reading-based question answering (QA), where evidence is explicitly provided or can be trivially retrieved.

arXiv AI

Jul 8

Data Analysis in the Wild: Benchmarking Large Language Models Against Real-World Data Complexities

arXiv:2607. 06482v1 Announce Type: cross Abstract: Current benchmarks for evaluating Large Language Models (LLMs) in data analysis often fail to reflect real-world settings.

By So Hasegawa, Shailaja Keyur Sampat, Lei Liu, Wei-Peng Chen

llms agents benchmarks

arXiv Machine Learning

Aug 12

SearchArt: Training Long-Horizon Search Agent with Scalable Synthetic and Verified Task

arXiv:2607. 24850v2 Announce Type: replace-cross Abstract: Recent advances in large language models (LLMs) have enabled search agents to autonomously tackle complex tasks across extended search and reasoning horizons.

By Lang Mei, Xiaohan Yu, Chong Chen, Liyan Liu, Xiangnan Chen, Jinchao Ma, Chao Feng, Li Huang, Siyu Mo, Sichen Kang, Yunkun Xu, Zhihan Yang, Zhujun Xue, Jingren Zhang, Qing He, Yingdi Huang, Hao Jiang, Ziao Ma, Zewei Pan, Minhao Sun, Zhuo Tao, Jinzhao Xiao, Gangtao Xin, Huanyao Zhang, Wenjian Zhang, Jiangshan Zhang, Guojie Zhu, Fangzhou Zou, Jiaxin Mao, Wentao Zhang

llms agents reinforcement-learning fine-tuning benchmarks

arXiv Computation and Language

Sep 10

Benchmarking Hybrid Deep Research Across Database Querying and Web Search

arXiv:2609.09410v1 Announce Type: new Abstract: While autonomous agents have made significant strides in "deep research" by iteratively navigating the open web to synthesize information, real-world p...

By Ruofan Wu, Peiran Xu, Xiaolong Li, Fan Shu, Soyoung Yoon, Yite Wang, Xiaodong Yu, Boyi Liu, Feng Yan, Debiao Li, Yuxiong He, Zhewei Yao

llms agents benchmarks data-engineering

arXiv Machine Learning

Jul 31

EMBL AI Librarian: Life-Sciences Knowledge Layer for AI Agents

arXiv:2607. 28229v1 Announce Type: cross Abstract: The web is increasingly accessed by AI agents rather than humans.

By Luigi Sigillo, Matteo Silvestri, Francesco Tabaro, Rajat Bhatnagar, Syed Irtaza Mubashar, Matt Jeffryes, Daljit Nijjer, Vittorio Perera, Ola Spjuth, Julio Saez-Rodriguez, Melissa Harrison, Fabio Petroni

llms agents nlp robotics benchmarks

arXiv AI

Jul 15

LakeQuest: A Three-Domain Benchmark for Grounded Question Answering across Data Lakes

arXiv:2607. 12310v1 Announce Type: cross Abstract: While modern question answering (QA) systems excel on clean, schema-aligned corpora, real-world knowledge is rarely so neatly packaged.

By Michael Solodko, Steven Gong, Guangwei Yu, Satya Krishna Gorti, Jesse C. Cresswell, Victor Zhong

rag agents nlp multimodal benchmarks

arXiv AI

Jun 6

Reducing Hallucinations in Complex Question Answering using Simple Graph-based Retrieval-Augmented Generation (long version)

arXiv:2606. 05901v1 Announce Type: cross Abstract: Large language models (LLMs) have fundamentally transformed the landscape of Natural Language Processing.

By Christopher J. Wedge, Joshua Stutter, Danny Dixon, Jacek Ca{\l}a

llms rag agents nlp fine-tuning benchmarks safety

arXiv Computation and Language

Sep 15

Where to Look and What to Use: Retrieve-Localize-Generate for Long-Term Conversational Memory Question Answering

arXiv:2609.07093v2 Announce Type: replace Abstract: Retrieval-augmented generation (RAG) enables large language models (LLMs) to answer questions by accessing external knowledge and has been widely a...

By Yifan Wang, Xinkui Lin, Yongxiu Xu, Shen Gao, Ruochen Yang, Kun Huang, Yubin Wang, Jie Wu, Wei Liu, Jian Luan, Hongbo Xu, Shuo Shang

llms rag nlp benchmarks

arXiv AI

Sep 15

Question's Gambit: The First Move Matters in Agentic Deep Search

arXiv:2609.14412v1 Announce Type: new Abstract: Deep research agents answer complex questions through iterative loops of searching, reading, and reasoning. Recent work on reasoning-intensive benchmar...

By Radin Hamidi Rad, Amin Bigdeli, Negar Arabzadeh, Sajad Ebrahimi, Charles L. A. Clarke, Benjamin C. M. Fung, Ebrahim Bagheri

llms rag agents benchmarks

arXiv AI

Aug 5

DocTrace: Towards Traceable Long Document VQA via Hierarchical Evidence Graph Reasoning

arXiv:2608. 03292v1 Announce Type: new Abstract: Long Document Visual Question Answering (LongDocVQA) requires Multimodal Large Language Models (MLLMs) to locate, integrate, and reason over heterogeneous document elements distributed across multiple pages.

By Le Xiang, Zhicheng Guan, Hong Chen, Xiaocong Lin, Zhenghua Lei, Teng Hu, Bolei He, Long Zeng

llms rag agents nlp fine-tuning multimodal benchmarks

arXiv AI

Jul 14

STEC: Evidence Compression for Deep Search in Open-domain Multi-Hop QA

arXiv:2607. 10795v1 Announce Type: new Abstract: In open-domain multi-hop question answering (QA), LLM-based search agents offer a promising approach to knowledge-intensive QA by combining retrieval with reasoning.

By Xinkang Li, Rong Jiang, Xin Song, Ye Wang, Yue Han, Changjian Li

llms agents nlp benchmarks

arXiv AI

4d ago

PrimeSeeker: Capability-Oriented Supervision for Deep Search Agents

arXiv:2609.35816v1 Announce Type: cross Abstract: Large language model search agents are often trained with synthetic questions whose difficulty is increased through larger evidence graphs, additiona...

By Linzhi Peng, Hanting Chen, Heng Chang, Ke Cheng, Bowen Du, Weifeng Lv

llms agents fine-tuning benchmarks

arXiv AI

Jul 31

SimpleWikiSearch: A Clean Offline Wikipedia Environment for Agentic Search

arXiv:2607. 26070v1 Announce Type: cross Abstract: Large language model (LLM)-based agentic search systems are often evaluated as if the underlying LLM were the only component that matters, yet their measured performance also depends on the surrounding search environment: the Wikipedia snapshot, preprocessing pipeline, chunking policy, retrieval backend, tool schema, observation format, and answer submission rule.

By Guanming Xiong, Penghui Zhang

llms rag agents

Related stories