CorporateBench (CB) is a large‑scale, human‑validated Q&A benchmark designed to evaluate large language models on enterprise‑scale document collections. It contains over 230,000 documents derived from four synthetically generated firms, each modeled with a temporally evolving knowledge base that ensures logical consistency across hundreds of thousands of documents. The benchmark tests LLMs on information extraction and knowledge‑base querying, revealing that performance degrades as input size approaches realistic corporate scales.
Enterprise question answering is framed as retrieving internal documents and generating grounded answers. Routine enterprise records, however, are work by-products in which required organizational relations remain implicit across heterogeneous sources.
arXiv:2607. 06482v1 Announce Type: cross Abstract: Current benchmarks for evaluating Large Language Models (LLMs) in data analysis often fail to reflect real-world settings.
arXiv:2608. 10679v1 Announce Type: cross Abstract: Enterprise question answering is framed as retrieving internal documents and generating grounded answers.
arXiv:2606. 04442v1 Announce Type: cross Abstract: AI systems increasingly need to combine two demanding capabilities: navigating multi-session conversation history and performing deep reading comprehension within long documents.
The paper introduces ElephantBench, a closed‑book knowledge probe with 1,094 multi‑account factual questions generated via an auditable graph‑based pipeline that pulls documents from a low‑exposure web corpus and identifies naturally occurring disagreements. Across 32 large language models, even the best model only recovers both divergent accounts on 52.4% of questions, and most models recall one account while omitting the other, indicating persistent epistemic myopia. The study shows that scaling model size and inference‑time reasoning improves recall but does not eliminate incompleteness, and that exposure imbalance in the corpus biases models toward the dominant account.
arXiv:2606. 10460v1 Announce Type: cross Abstract: Recent large language models (LLMs) have shown rapid progress in reading-based question answering (QA), where evidence is explicitly provided or can be trivially retrieved.
AI systems increasingly need to combine two demanding capabilities: navigating multi-session conversation history and performing deep reading comprehension within long documents. Yet no existing benchmark evaluates both simultaneously.
arXiv:2411. 19504v2 Announce Type: replace Abstract: The advance of large language models (LLMs) has unlocked great opportunities in complex multi-modal data management tasks, particularly in question answering (QA) over complicated multi-table relational data.
Corpus2Skill is a retrieval architecture that transforms an enterprise knowledge base into a hierarchical skill directory, enabling an LLM agent to navigate from high-level summaries to specific documents and backtrack when necessary. On an enterprise customer‑support benchmark, it outperforms single‑shot dense, hybrid, hierarchical‑retrieval, and agentic RAG baselines in answer quality and grounding, with a moderate cost tradeoff. An eleven‑dataset study shows that corpus navigation excels on single‑domain corpora with a recoverable topical taxonomy but is less effective on open‑domain factoid pools or homogeneous‑tabular corpora, providing a design guideline for knowledge‑grounded systems.
arXiv:2608. 14747v1 Announce Type: new Abstract: WANDR (Wide ANd Deep Research) is a benchmark of 500 realistic, challenging data-collection tasks for research agents.
arXiv:2606. 27742v1 Announce Type: cross Abstract: Enterprise Knowledge Graphs (KGs) are increasingly used for internal search, analytics, and question answering, but building natural-language interfaces for private enterprise graphs remains costly.
arXiv:2607. 29677v1 Announce Type: new Abstract: Enterprise workflows increasingly rely on agents for \emph{schema-guided extraction}: given a document and a user-defined schema, the agent faithfully follows the schema to produce the correct output with source evidence as grounding metadata.