arXiv AI
Sep 2

Verifiable Disaster Storylines and Causal Knowledge Graphs: A Citation-Grounded Pipeline from Heterogeneous Humanitarian Sources

The paper introduces a pipeline that merges structured disaster records from EM‑DAT with unstructured documents from ReliefWeb and the European Media Monitor to generate source‑grounded disaster storylines and causal knowledge graphs. Using Retrieval‑Augmented Generation, it produces tabular event profiles covering 17 fields and builds causal graphs enriched with citation‑grounded explanatory narratives, allowing traceability to primary sources. Human evaluation across three crisis cases shows high retrieval precision, strong faithfulness of causal relations, and a clear expert preference for citation‑grounded components over ungrounded ones.

By Ivan Decostanzi, Michele Ronco, Sergio Consoli, Christina Corbane, Lorenzo Bertolini, Indaco Biazzo, Daria Mihaila, Manuel Garcia-Herranz, Felix Schwebel, Yelena Mejova, Kyriaki Kalimeri
arXiv Machine Learning
Jul 27

SCOPE and SCION: A Benchmark and an Auditable Reference Pipeline for Schema Induction and Fusion from Text

arXiv:2607. 21610v1 Announce Type: cross Abstract: Schema graphs are an upstream bottleneck of schema-grounded information extraction and knowledge graph construction, yet most extraction systems assume the schema is already available.

By Miaobo Hu, Xiaobo Guo, Shuhao Hu, Bokun Wang, Rui Chen, Xin Wang, Daren Zha, Jun Xiao
arXiv Computation and Language
3d ago

TRACE: Target-Aware Retrieval, Attributed Evidence, and Contract-Constrained Extraction for LitTraceQA

TRACE is a system designed to bridge the grounding contract gap in LitTraceQA by combining target-aware retrieval, independent typed evidence localization, multimodal table extraction, and schema-driven table construction. It indexes 27,487 papers using multiple representations while preserving question targets, predicts observation units for tables, and assembles rows with evaluator-compatible key normalization. On the official test set, TRACE achieves a 0.760613 overall score, with high paper F1, evidence F1, and multiple-choice accuracy, though table-row and macro cell performance remain lower.

By Sachin Gupta, Divya Godara
arXiv Computation and Language
Sep 11

RAG-Safety-Bench: Reliable Evaluation of Retrieval-Augmented LLM Safety

RAG-Safety-Bench is a benchmark designed to evaluate how retrieval-augmented generation (RAG) affects the safety of large language models (LLMs). It isolates safety impacts by testing four conditions: non-RAG, RAG with an oracle document, RAG with related but non-answer documents, and RAG with random safe documents. Results on five open-source LLMs reveal an inverse relationship between benign and unsafe capabilities, show that baseline safety guardrails do not guarantee safety in RAG, and confirm that even benign documents can trigger unsafe generation.

By Adithiyan Rajan Indira Saravanan, Kathleen C. Fraser
Hugging Face Trending Papers
Sep 10

RAG-Safety-Bench: Reliable Evaluation of Retrieval-Augmented LLM Safety

RAG-Safety-Bench is a benchmark designed to evaluate how retrieval-augmented generation (RAG) affects the safety of large language models (LLMs). It isolates safety impacts by testing four conditions: non-RAG, RAG with an oracle document, RAG with related but non-answer documents, and RAG with random safe documents. Results on five open-source LLMs reveal an inverse relationship between benign and unsafe capabilities, show that standard safety guardrails do not guarantee safety in RAG, and confirm that even benign documents can trigger unsafe outputs.