arXiv Computation and Language

CMNIE: An Information Extraction Benchmark for Chinese Military News

CMNIE is a new benchmark for extracting structured information from Chinese military news, covering event triggers, arguments, named entities, and entity relations under a unified schema. The dataset contains 13,000 manually annotated instances with 7 event types, 10 argument roles, 7 entity types, and 8 relation types. Experiments show that current supervised models, zero‑shot LLMs, and fine‑tuned LLMs struggle with relation extraction and exact span matching, highlighting the challenge of joint structured extraction in this domain.

arXiv Computation and Language
Sep 16

SciNLP: A Domain-Specific Benchmark for Full-Text Scientific Entity and Relation Extraction in NLP

SciNLP is a new benchmark dataset for full‑text entity and relation extraction in the NLP domain, comprising 60 manually annotated papers with 6,429 entities and 1,649 relations. It is the first dataset to provide full‑text annotations of entities and their relationships specifically for NLP literature. Experiments show that models trained on SciNLP outperform baselines on certain tasks, and the dataset enabled the automatic construction of a fine‑grained knowledge graph with an average node degree of 3.3.

By Decheng Duan, Yingyi Zhang, Jitong Peng, Chengzhi Zhang
arXiv Computation and Language
Sep 14

Extracting Dataset Mentions in Forced Displacement and FCV Documents: A Weakly Supervised Framework with LLM-Based Label Refinement

The paper introduces a weakly supervised framework for extracting dataset mentions from forced displacement and Fragile, Conflict, and Violence (FCV) documents. It uses a lightweight model trained on general research literature to generate candidate mentions, which are then refined by a large language model that validates or rejects them and corrects boundaries. The refined annotations are augmented with synthetic and contrastive examples to fine‑tune the model, achieving 74.1% precision and 70.5% recall on a benchmark of 1,706 passages, with higher precision (89.5%) on passages that contain dataset references.

By Rafael Macalaba, Aivin V. Solatorio, Patrick Michael Brock, Olivier Dupriez
Hugging Face Trending Papers
Jul 30

From Single- to Cross-Document: Benchmarking Multi-Granularity Event Analysis of Large Language Models

Event analysis is an essential and fundamental direction of information extraction, involving various event-centric tasks at different granularity of documents. While large language models (LLMs) have preliminarily achieved promising performance in part of these tasks individually, their capability in event analysis still lacks comprehensive understanding due to restricted document granularity, task designs, and data source of existing benchmarks.

arXiv Computation and Language
Sep 3

CHisAgent: A Multi-Agent Framework for Event Taxonomy Construction in Ancient Chinese Cultural Systems

CHisAgent is a multi‑agent large language model framework designed to build event taxonomies for ancient Chinese cultural contexts. It splits the task into three specialized stages: an Inducer that creates an initial hierarchy from raw historical texts, an Expander that adds missing intermediate concepts using LLM world knowledge, and an Enricher that incorporates external structured historical resources to ensure faithfulness. Applied to the Twenty‑Four Histories, CHisAgent produced a comprehensive, domain‑aware taxonomy covering politics, military, diplomacy, and social life, and evaluations show improved structural coherence, coverage, and cross‑cultural alignment.

By Xuemei Tang, Chengxi Yan, Jinghang Gu, Chu-Ren Huang
arXiv Machine Learning
Jul 27

SCOPE and SCION: A Benchmark and an Auditable Reference Pipeline for Schema Induction and Fusion from Text

arXiv:2607. 21610v1 Announce Type: cross Abstract: Schema graphs are an upstream bottleneck of schema-grounded information extraction and knowledge graph construction, yet most extraction systems assume the schema is already available.

By Miaobo Hu, Xiaobo Guo, Shuhao Hu, Bokun Wang, Rui Chen, Xin Wang, Daren Zha, Jun Xiao
arXiv AI
Aug 18

A Large-Scale Chinese Knowledge Graph-Text Alignment Dataset for Benchmarking Knowledge-Grounded LLMs

arXiv:2510. 06039v2 Announce Type: replace-cross Abstract: Reliable evaluation of knowledge-grounded Large Language Models (LLMs) in Chinese requires resources that explicitly align Chinese-language text with verifiable Knowledge Graph (KG) facts.

By Chengwei Wu, Xingrui Zhuo, Mingyang Gao, Xinghe Cheng, Zhichao Yan, Jiapu Wang
arXiv Computation and Language
Sep 1

Configurable Semantic Chunking for Biomedical Information Extraction in Retrieval-Augmented Generation

The paper introduces a configurable semantic chunking framework for biomedical information extraction in retrieval‑augmented generation systems. It replaces the fixed‑size chunking stage of BioMedRAG with entity‑preserving windows, trigger‑centered chunking, proposition‑first extraction, tiered trigger prioritization, and hierarchical relation resolution, while keeping the rest of the pipeline unchanged. Experiments on relation extraction benchmarks (GM‑CIHT, DDI, ChemProt) and adverse event classification (ADE) show that the hybrid configuration boosts performance on datasets with explicit relation cues, achieving 82.6% F1 on GM‑CIHT compared to 74.2% with the baseline.

By Riya Ahuja (Institute of Data Science in Biomedicine, TU Braunschweig, Braunschweig, Germany, Braunschweig Integrated Centre of Systems Biology, TU Braunschweig, Braunschweig, Germany), Tim Kacprowski (Institute of Data Science in Biomedicine, TU Braunschweig, Braunschweig, Germany, Braunschweig Integrated Centre of Systems Biology, TU Braunschweig, Braunschweig, Germany), Roya Shiasi Sardoabi (Institute of Data Science in Biomedicine, TU Braunschweig, Braunschweig, Germany, Braunschweig Integrated Centre of Systems Biology, TU Braunschweig, Braunschweig, Germany)
arXiv Computation and Language
Aug 25

A Scalable Cross-Domain Event Extraction System via a Unified Generative Training Framework

The paper introduces a scalable cross‑domain event extraction system built on a unified generative sequence‑to‑sequence framework. It jointly handles event detection and argument extraction, allowing both pipeline and end‑to‑end configurations. By fine‑tuning pretrained language models on multiple event datasets from diverse domains, the system retains domain‑specific semantics while generalizing across large, evolving label spaces, and offers a web‑based application for researchers to upload documents, extract events, visualize triggers and arguments, and compare configurations.

By Siting Liang, Omar Adjali, Omair Shahzad Bhatti, Daniel Sonntag
arXiv Computation and Language
Aug 25

A Multi-Domain and Multi-Task Generative Framework with Explicit Task and Domain Conditioning for Cross-Domain Event Extraction

The paper introduces a unified multi-domain and multi-task generative framework for event extraction that incorporates explicit domain conditioning signals and task-specific prompts. This design allows a single model to adapt dynamically to different event schemas without needing full event label sets during inference, supporting both pipeline and end-to-end extraction. Experiments on various benchmarks show competitive performance, strong cross-domain generalization, and practical scalability while maintaining domain-specific precision.

By Siting Liang, Omar Adjali, Daniel Sonntag
arXiv AI
Sep 15

Enhancing Event Candidate Acquisition for Event Linking

The paper introduces MACE, a Multi-Agent Candidate Event acquisition method designed to improve event linking by refining event structure before the linking step. MACE employs evidence-specialized large language model agents to gather time, location, participant, and event-type evidence, exposes intermediate queries to candidate-event lookup tools, and allows a coordinator to revise the evidence set before final candidate construction. Experiments on two event linking benchmarks demonstrate that integrating MACE consistently boosts accuracy across different event linking models without altering the underlying models.

By Ziyang Zhang, Yinan Liu, Boyi Xue, Yingxuan Huang, Bin Wang, Xiaochun Yang