arXiv AI

Do Data Agents Need Semantic Metadata? A Comparative Study in Agentic Data Retrieval

arXiv:2605. 28787v2 Announce Type: replace-cross Abstract: In the era of autonomous agents, machine-actionable data is critical for data-driven workflows.

arXiv AI
Sep 15

EvoOntology: A Self-Evolving Ontology Layer for Data Agents

EvoOntology introduces a self‑evolving ontology layer for data agents, encapsulating the ontology as an MCP server with schema, content, and tool layers. It enables agents to query and interact with the ontology at runtime, using a builder agent for autonomous construction and a self‑evolution loop that refines the ontology through attribution‑guided edits validated by backbone‑conditional evaluation. Experiments on three data‑agent benchmarks with four LLM backbones show that EvoOntology consistently outperforms strong baselines and existing semantic‑layer approaches, effectively bridging the agent‑data gap for heterogeneous data.

By Meiduo Chong, Shaolei Zhang, Ju Fan, Xiaoyong Du
arXiv Computation and Language
Sep 10

Benchmarking Hybrid Deep Research Across Database Querying and Web Search

arXiv:2609.09410v1 Announce Type: new Abstract: While autonomous agents have made significant strides in "deep research" by iteratively navigating the open web to synthesize information, real-world p...

By Ruofan Wu, Peiran Xu, Xiaolong Li, Fan Shu, Soyoung Yoon, Yite Wang, Xiaodong Yu, Boyi Liu, Feng Yan, Debiao Li, Yuxiong He, Zhewei Yao
arXiv AI
Sep 10

DI-Bench: Systematically Generating In-Domain Data Intelligence Benchmarks for Enterprise Agents

DI-Bench is a pipeline that automatically creates realistic data intelligence benchmarks for enterprise agents by linking data tables, dimensions, metrics, and documents into an artifact graph. It generates questions that combine structured data queries with knowledge retrieval, validates answers via query execution and LLM-generated questions, and has produced a 731-task benchmark covering knowledge retrieval, analytical computation, and rule‑grounded reasoning. Evaluation of four models on this benchmark shows that only 32% accuracy is achieved on computational tasks that involve business rules modifying the computation.

By Jiangyun Zhang, Kristen Surrao, Torpong Nitayanont, Yupei Zhang, Roopali Singh, Zhiyu Chen, Julia Huang, Zhou Tang, Shayan Ali Akbar, Omar Alonso, Erwin Cornejo, Yuan Li, Yi Zhang
arXiv AI
Aug 5

DataSpace: Benchmarking Data Agents for Verifiable Analytics over Heterogeneous Workspaces

arXiv:2608. 03451v1 Announce Type: new Abstract: Data agents enable natural-language analytics over organizational workspaces, where relevant evidence may be scattered across databases, structured files, long documents, and multimedia.

By Boyan Li, Zhuowen Liang, Yupeng Xie, Xiaotian Lin, Tianqi Luo, Xinyu Liu, Yizhang Zhu, Zhangyang Peng, Yuan Li, Zhengxuan Zhang, Jiayi Zhang, Nan Tang, Guoliang Li, Yuyu Luo