arXiv AI By Shuai Yan, Yuhang Wu, Xiaodong Huang, Ke Wang

Decoupling Error Attribution in Cloud-Native Graph-RAG: A Data Integrity Diagnostic Framework

Read the original on arXiv AI →

The paper introduces a three‑layer diagnostic framework that separates errors in Graph‑RAG systems into reasoning loss, Knowledge Graph defects, and Cypher generation errors. Experiments on a spatio‑temporal ecological KG from southeastern Tibet show that data integrity issues, especially structural defects, are the main cause of performance drops, reducing accuracy from 0.93 to 0.39. The study also identifies a Parametric Knowledge Masking Effect where LLMs hide retrieval path failures, masking over 70 % of query generation errors and raising the risk of false negatives in automated monitoring.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language
Aug 27

Trace Integrity for LLM Data Agents: A Vision for Auditable Structured Reasoning in Real-World Systems

The paper argues that answer accuracy alone is insufficient for evaluating large language model (LLM) data agents, especially in structured-data tasks where a correct answer can be produced by an invalid trace. It introduces Trace Integrity as a reliability criterion that ensures the computation behind an answer is explicit, executable, schema-valid, operator-faithful, replayable, answer-consistent, and auditable. The authors operationalize this concept with execution contracts and present the CAIT (Correct Answer / Invalid Trace) Rate to quantify how often answer-only evaluations mistakenly reward unsupported outputs, demonstrating that accuracy, trace validity, and silent-failure risk are distinct signals.

By Srimonti Dutta, Akshata Kishore Moharir