The paper introduces the DevRev NL2SQL benchmark, comprising 900 execution‑verified queries that feature nested‑type and link‑graph structures, along with the Semantic Depth Score (SDS) to assess analytical reasoning depth. It also presents a cost‑aware single‑generation agentic architecture that includes schema selection, metadata retrieval, and error‑repair components tailored to nested enterprise schemas. On the DevRev benchmark, the system achieves 91.7% answer correctness, outperforming the next‑best baseline by 54.6 percentage points, and remains competitive on the Spider 2.0 Snowflake dataset.
The paper introduces the DevRev NL2SQL benchmark, featuring 900 execution‑verified queries that test natural‑language‑to‑SQL systems on nested, graph‑like enterprise schemas, and proposes the Semantic Depth Score (SDS) as a rubric for analytical reasoning depth. It also presents a cost‑aware, single‑generation agentic architecture that includes schema selection, metadata retrieval, and error‑repair components tailored to these complex schemas. On the DevRev benchmark, the system achieves 91.7% answer correctness, outperforming the next‑best baseline by 54.6 percentage points, and remains competitive on the Spider 2.0 Snowflake dataset.
By Yoga Sri Varshan Varadharajan, Ajay Yadav, Ritesh Goru, Prateek Chaudhury, Constantine Caramanis, Prateek Jain, Divyateja Pasupuleti, Sunil Kumar Pandey
arXiv:2609.13808v1 Announce Type: new
Abstract: Structured knowledge fact checking aims to determine the truthfulness of natural language claims by reasoning over structured evidence. Recent program-...
By Yifei Li, Xiaohan Zheng, Wentao Qian, Liansheng Zhuang
arXiv:2607. 22713v1 Announce Type: new Abstract: Enterprise IT support knowledge graphs capture rich relationships among cases, users, devices, symptoms, taxonomic categories, root causes, and historical resolutions.
By Saiyue Lyu, Mariam Dundua, Vishaal Kapoor, Sarthak Ahuja, Neda Kordjazi, Evren Yortucboylu, Harsh Amin, Rebecca Steinert
arXiv:2605.14259v3 Announce Type: replace
Abstract: Applying Large Language Models (LLMs) to heterogeneous enterprise systems is hindered by hallucinations and failures in multi-hop, n-ary reasoning....
By Ling Wang, Xin Liu, Songnan Liu, Jianan Wang, Cheng Cheng, Yihan Zhu, Enyu Li, Yu Xiao, Jiangyong Xie, Duogong Yan, Jiangyi Chen
arXiv:2608. 15834v1 Announce Type: new Abstract: Tool-calling LLM agents navigate unfamiliar codebases with a handful of generic primitives for listing, reading and searching files (ls, cat, grep).
By Marius Dragic, Ruben Ifrah, Alexandre Rio
arXiv:2601. 21162v2 Announce Type: replace-cross Abstract: Graph Retrieval-Augmented Generation (Graph-RAG) enhances multihop question answering by organizing corpora into knowledge graphs and routing evidence through relational structure.
By Jiate Liu, Zebin Chen, Shaobo Qiao, Mingchen Ju, Danting Zhang, Bocheng Han, Shuyue Yu, Xin Shu, Jinglin Wu, Dong Wen, Xin Cao, Guanfeng Liu, Zhengyi Yang
DI-Bench is a pipeline that automatically creates realistic data intelligence benchmarks for enterprise agents by linking data tables, dimensions, metrics, and documents into an artifact graph. It generates questions that combine structured data queries with knowledge retrieval, validates answers via query execution and LLM-generated questions, and has produced a 731-task benchmark covering knowledge retrieval, analytical computation, and rule‑grounded reasoning. Evaluation of four models on this benchmark shows that only 32% accuracy is achieved on computational tasks that involve business rules modifying the computation.
By Jiangyun Zhang, Kristen Surrao, Torpong Nitayanont, Yupei Zhang, Roopali Singh, Zhiyu Chen, Julia Huang, Zhou Tang, Shayan Ali Akbar, Omar Alonso, Erwin Cornejo, Yuan Li, Yi Zhang
arXiv:2608.29263v1 Announce Type: new
Abstract: Large Language Models (LLMs) often suffer from hallucination and struggle with complex reasoning tasks requiring multi-hop domain knowledge. While inte...
By Yuwei Lou, Hao Hu, Yuzhou Jiang, Zongfei Zhang, Liang Wang, Jincai Liu, Jidong Ge, Xianping Tao
arXiv:2608.22974v1 Announce Type: new
Abstract: Large language model (LLM) agents rely heavily on knowledge encoded in model parameters or presented as unstructured context. In domain-specific tasks,...
By Xiaohui Zhang, Zequn Sun, Chengyuan Yang, Yuanning Cui, Lingbing Guo, Wei Hu
The paper introduces WFM, a Wiki Foundation Model designed to support complex agentic reasoning by combining dense document contexts with markdown-based topological linkages. It formalizes a Wiki Graph schema that preserves explicit topologies while embedding continuous semantics, and employs a query‑conditioned attentive aggregation for efficient message passing. The authors also propose an NCCL‑based protocol to reduce distributed system overhead, achieving a 10.5× training speedup and strong performance on long‑term memory and multi‑hop reasoning benchmarks.
By Junnan Dong, Linhao Luo, Senlei Zhang, Gong Chen, Taian Guo, Yifei Yu, Rong Tao, Tao Guo, Qian-Wen Zhang, Siyu An, Ruizhi Qiao, Xing Sun
arXiv:2608. 02650v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly rely on external tools to complete complex real-world tasks.
By Zian Zhai, Xingyu Tan, Gaowang Zou, Xiaoyang Wang, Wenjie Zhang