CMNIE is a new benchmark for extracting structured information from Chinese military news, covering event triggers, arguments, named entities, and entity relations under a unified schema. The dataset contains 13,000 manually annotated instances with 7 event types, 10 argument roles, 7 entity types, and 8 relation types. Experiments show that current supervised models, zero‑shot LLMs, and fine‑tuned LLMs struggle with relation extraction and exact span matching, highlighting the challenge of joint structured extraction in this domain.
arXiv:2610.17113v1 Announce Type: cross Abstract: Information extraction models often perform well on the datasets they are trained on, but transferring to new domains and annotation guidelines remai...
SciNLP is a new benchmark dataset for full‑text entity and relation extraction in the NLP domain, comprising 60 manually annotated papers with 6,429 entities and 1,649 relations. It is the first dataset to provide full‑text annotations of entities and their relationships specifically for NLP literature. Experiments show that models trained on SciNLP outperform baselines on certain tasks, and the dataset enabled the automatic construction of a fine‑grained knowledge graph with an average node degree of 3.3.
The paper introduces a weakly supervised framework for extracting dataset mentions from forced displacement and Fragile, Conflict, and Violence (FCV) documents. It uses a lightweight model trained on general research literature to generate candidate mentions, which are then refined by a large language model that validates or rejects them and corrects boundaries. The refined annotations are augmented with synthetic and contrastive examples to fine‑tune the model, achieving 74.1% precision and 70.5% recall on a benchmark of 1,706 passages, with higher precision (89.5%) on passages that contain dataset references.
Event analysis is an essential and fundamental direction of information extraction, involving various event-centric tasks at different granularity of documents. While large language models (LLMs) have preliminarily achieved promising performance in part of these tasks individually, their capability in event analysis still lacks comprehensive understanding due to restricted document granularity, task designs, and data source of existing benchmarks.
CHisAgent is a multi‑agent large language model framework designed to build event taxonomies for ancient Chinese cultural contexts. It splits the task into three specialized stages: an Inducer that creates an initial hierarchy from raw historical texts, an Expander that adds missing intermediate concepts using LLM world knowledge, and an Enricher that incorporates external structured historical resources to ensure faithfulness. Applied to the Twenty‑Four Histories, CHisAgent produced a comprehensive, domain‑aware taxonomy covering politics, military, diplomacy, and social life, and evaluations show improved structural coherence, coverage, and cross‑cultural alignment.
arXiv:2607. 21610v1 Announce Type: cross Abstract: Schema graphs are an upstream bottleneck of schema-grounded information extraction and knowledge graph construction, yet most extraction systems assume the schema is already available.
arXiv:2510. 06039v2 Announce Type: replace-cross Abstract: Reliable evaluation of knowledge-grounded Large Language Models (LLMs) in Chinese requires resources that explicitly align Chinese-language text with verifiable Knowledge Graph (KG) facts.
The paper evaluates large pre‑trained language models (LLMs) for knowledge entity relation extraction (KERE) from academic articles, comparing multiple strategies such as closed‑source LLM CoT prompting, LoRA‑adapted open‑source LLMs, and various LLM‑enhanced small pre‑trained language models (SLMs). Two manually annotated datasets in the NLP domain—one for abstracts and one for full texts—were used, and the LoRA‑adapted T5 model achieved the highest macro F1 scores (0.7408 for abstracts and 0.7013 for full texts). Error analysis shows that LLM‑based strategies mitigate SLM errors differently across error types, and combining complementary strategies further improves KERE performance.
arXiv:2610.34343v1 Announce Type: cross Abstract: Structured semantic parsing is typically judged by whether the correct category is assigned, which leaves value recovery and instance-level associati...
arXiv:2606. 19710v1 Announce Type: cross Abstract: Court proceedings contain valuable evidence about human smuggling networks, but this information is often buried within unstructured, jargon-heavy legal documents.
Was this person ever at that place, and if so, when? Answering such questions from noisy, multilingual historical documents is the central challenge of HIPE-2026, the third edition of the HIPE evaluation series.
The paper introduces a configurable semantic chunking framework for biomedical information extraction in retrieval‑augmented generation systems. It replaces the fixed‑size chunking stage of BioMedRAG with entity‑preserving windows, trigger‑centered chunking, proposition‑first extraction, tiered trigger prioritization, and hierarchical relation resolution, while keeping the rest of the pipeline unchanged. Experiments on relation extraction benchmarks (GM‑CIHT, DDI, ChemProt) and adverse event classification (ADE) show that the hybrid configuration boosts performance on datasets with explicit relation cues, achieving 82.6% F1 on GM‑CIHT compared to 74.2% with the baseline.