arXiv AI

LakeQuest: A Three-Domain Benchmark for Grounded Question Answering across Data Lakes

arXiv:2607. 12310v1 Announce Type: cross Abstract: While modern question answering (QA) systems excel on clean, schema-aligned corpora, real-world knowledge is rarely so neatly packaged.

arXiv AI

Jun 10

LakeQA: An Exploratory QA Benchmark over a Million-Scale Data Lake

arXiv:2606. 10460v1 Announce Type: cross Abstract: Recent large language models (LLMs) have shown rapid progress in reading-based question answering (QA), where evidence is explicitly provided or can be trivially retrieved.

By Haonan Wang, Jiaxiang Liu, Yurong Liu, Austin Senna Wijaya, Tianle Zhou, Eden Wu, Yijia Chen, Wanting You, Reya Vir, Daniela Pinto, Grace Fan, Yusen Zhang, Juliana Freire, Eugene Wu

llms agents nlp benchmarks

arXiv AI

Jul 8

From Voting to Agent Collaboration: Answer-Type-Aware LLM Pipelines for BioASQ 14b

arXiv:2607. 06452v1 Announce Type: cross Abstract: Biomedical question answering requires not only accurate extraction of information from scientific literature but also reliable integration of evidence across multiple documents.

By Taeyun Roh, Eunha Lee, Wonjune Jang, Sohyun Chung, Junha Jung, Jaewoo Kang

llms agents nlp

arXiv AI

Aug 3

Can Large Language Models Derive New Knowledge? A Dynamic Benchmark for Biological Knowledge Discovery

arXiv:2603. 03322v2 Announce Type: replace-cross Abstract: Recent advancements in Large Language Model (LLM) agents have demonstrated remarkable potential in automatic knowledge discovery.

By Chaoqun Yang, Xinyu Lin, Shulin Li, Wenjie Wang, Ruihan Guo, Fuli Feng, Tat-Seng Chua

llms agents benchmarks

arXiv AI

Jul 15

CANDI: Contextual Alignment for Niche Domains Question Answering

arXiv:2607. 11891v1 Announce Type: cross Abstract: The deployment of large language models (LLMs) in specialized domains like medical diagnostics and financial advisory necessitates evaluating capabilities beyond general knowledge.

By Megha Chakraborty, Darssan L. Eswaramoorthi, Het Riteshkumar Shah, Madhur Thareja, Michelle A Ihetu, Harshul Raj Surana, Kaushik Roy, Amit Sheth

llms nlp benchmarks safety

arXiv AI

Jun 9

TQA-Bench: Evaluating LLMs for Multi-Table Question Answering

arXiv:2411. 19504v2 Announce Type: replace Abstract: The advance of large language models (LLMs) has unlocked great opportunities in complex multi-modal data management tasks, particularly in question answering (QA) over complicated multi-table relational data.

By Zipeng Qiu, Chenyue Li, You Peng, Guangxin He, Binhang Yuan, Chen Wang

llms nlp multimodal benchmarks

arXiv Machine Learning

2d ago

AutoSchema: Live Schema Grounding for Agentic Text-to-Sparql over Heterogeneous Knowledge Graphs

arXiv:2608. 14228v1 Announce Type: new Abstract: Life science knowledge graphs make large collections of structured data available through SPARQL, but each resource uses its own schema, identifiers, and links.

By Yiming Zhang, Koji Tsuda

llms agents

arXiv AI

Jul 14

STEC: Evidence Compression for Deep Search in Open-domain Multi-Hop QA

arXiv:2607. 10795v1 Announce Type: new Abstract: In open-domain multi-hop question answering (QA), LLM-based search agents offer a promising approach to knowledge-intensive QA by combining retrieval with reasoning.

By Xinkang Li, Rong Jiang, Xin Song, Ye Wang, Yue Han, Changjian Li

llms agents nlp benchmarks

arXiv Machine Learning

Jun 5

Executable Schema Contracts: From Automatic Ingestion to Multi-Source Retrieval

arXiv:2606. 05415v1 Announce Type: cross Abstract: Real-world data spans tables, documents, and semi-structured files with implicit semantics.

By Padmaja Jonnalagedda, Yuguang Yao, Xiang Gao, Hilaf Hasson, Kamalika Das

llms rag agents benchmarks

arXiv AI

Jun 30

DEEPMED Search: An Open-Source Agentic Platform for Medical Deep Research with Introspective Verification

arXiv:2606. 29746v1 Announce Type: new Abstract: Navigating the deluge of heterogeneous medical data, from academic literature (PubMed) to clinical guidelines (Web) and private knowledge bases, remains a critical bottleneck for evidence-based medicine.

By Maolin Liu, Fanyu Xu, Ruoqing Xu, Jiahang Zhang, Hao Wang, Rui Wang

rag agents

arXiv AI

Jul 9

Health System Scale Semantic Search Across Unstructured Clinical Notes

arXiv:2604. 25605v2 Announce Type: replace-cross Abstract: Introduction: Semantic search, which retrieves documents based on conceptual similarity rather than keywords, offers advantages for retrieval of clinical information.

By Faith Wavinya Mutinda, Spandana Makeneni, Anna Lin, Shivaji Dutta, Irit R. Rasooly, Patrick Dibussolo, Shivani Kamath Belman, Hessam Shahriari, Kevin Murphy, Alex B. Ruan, Barbara H. Chaiyachati, Sanjay Chainani, Robert W. Grundmeier, Scott M. Haag, Jeffrey M. Miller, Heather M. Griffis, Ian M. Campbell

llms rag benchmarks

arXiv AI

Jun 26

ProvenAI: Provenance-Native Traces of Evidence in Generated Answers

arXiv:2606. 26449v1 Announce Type: cross Abstract: Retrieval-augmented systems routinely present citations alongside generated answers, yet a citation does not confirm that the corresponding source meaningfully shaped the output.

By Mohammad Faizan, Dalal Alharthi

rag agents nlp benchmarks

arXiv AI

Jul 24

SciExplore: Evaluating Autonomous Agents from Scientific Navigation to Information Integration

arXiv:2607. 20926v1 Announce Type: new Abstract: Scientific research involves complex information-seeking and reasoning workflows across heterogeneous sources.

By Yinhao Tang, Youqing Fang, Yanan Sun, Wenran Liu, Weiming Zhang, Bin Liu, Kuikun Liu, Wenwei Zhang, Kai Chen

llms agents nlp benchmarks

Related stories