arXiv Computation and Language

SciNLP: A Domain-Specific Benchmark for Full-Text Scientific Entity and Relation Extraction in NLP

SciNLP is a new benchmark dataset for full‑text entity and relation extraction in the NLP domain, comprising 60 manually annotated papers with 6,429 entities and 1,649 relations. It is the first dataset to provide full‑text annotations of entities and their relationships specifically for NLP literature. Experiments show that models trained on SciNLP outperform baselines on certain tasks, and the dataset enabled the automatic construction of a fine‑grained knowledge graph with an average node degree of 3.3.

arXiv Computation and Language
5d ago

Enhancing Knowledge Entity Relation Extraction in Abstracts and Full-text Content of Academic Articles Using Large Language Models

The paper evaluates large pre‑trained language models (LLMs) for knowledge entity relation extraction (KERE) from academic articles, comparing multiple strategies such as closed‑source LLM CoT prompting, LoRA‑adapted open‑source LLMs, and various LLM‑enhanced small pre‑trained language models (SLMs). Two manually annotated datasets in the NLP domain—one for abstracts and one for full texts—were used, and the LoRA‑adapted T5 model achieved the highest macro F1 scores (0.7408 for abstracts and 0.7013 for full texts). Error analysis shows that LLM‑based strategies mitigate SLM errors differently across error types, and combining complementary strategies further improves KERE performance.

By Yingyi Zhang, Chengzhi Zhang
arXiv Computation and Language
Sep 23

Domain-Adaptive Pretraining Enhances Water Treatment Semantic Representation for Large-Scale Structured Literature Mining

The paper introduces WaterBERT, a domain‑adapted encoder model trained on a 2.97‑billion‑token water treatment corpus to capture domain‑specific semantics for literature mining. Fine‑tuned versions of WaterBERT outperform general‑purpose and other domain BERT models on tasks such as treatment process classification, named entity recognition, and relation extraction. The authors also demonstrate WaterBERT’s utility in large‑scale processing, generating coherent research topics, building a structured knowledge graph from 693,211 abstracts, and creating a Water Knowledge‑Enhanced Retrieval System that surpasses text‑based baselines.

By Mudi Zhai (UNSW Water Research Centre, School of Civil and Environmental Engineering, The University of New South Wales, Sydney, NSW 2052, Australia), Ruihong Qiu (School of Electrical Engineering and Computer Science, The University of Queensland, Brisbane, QLD 4072, Australia), Qingyun Zeng (Microsoft Copilot Studio AI, Redmond, WA 98052, United States, Departments of Mathematics & Department of Computer and Information Science, University of Pennsylvania, Philadelphia, PA 19104, United States), T. David Waite (UNSW Water Research Centre, School of Civil and Environmental Engineering, The University of New South Wales, Sydney, NSW 2052, Australia), Bing-Jie Ni (UNSW Water Research Centre, School of Civil and Environmental Engineering, The University of New South Wales, Sydney, NSW 2052, Australia), Haoran Duan (UNSW Water Research Centre, School of Civil and Environmental Engineering, The University of New South Wales, Sydney, NSW 2052, Australia, Department of Civil Engineering, The University of Hong Kong, Pokfulam, Hong Kong SAR, China)
arXiv Computation and Language
Sep 22

Custom Named Entity Recognition and Topic Classification for Global Health Publications

This thesis explores how to select and adapt NLP models for global health literature when annotated data and computational resources are scarce. It compares skip‑gram word2vec models trained on increasingly large specialized corpora with BioWordVec for semantic tag discovery, finding that larger coverage does not always yield more useful domain associations. The study also evaluates convolutional spaCy models versus a RoBERTa transformer for named entity recognition, noting a trade‑off between higher F1 scores and longer inference time, and investigates MiniLM few‑shot versus BART‑MNLI zero‑shot classification for multi‑label topic classification, highlighting practical constraints of inference cost. "whyItMatters":"The work provides empirical guidance on balancing model accuracy and resource demands for building knowledge systems in low‑resource global health settings."

By Genis Skura, Antoine Geissb\"uhler, Jean-Luc Falcone
arXiv Computation and Language
Sep 11

CMNIE: An Information Extraction Benchmark for Chinese Military News

CMNIE is a new benchmark for extracting structured information from Chinese military news, covering event triggers, arguments, named entities, and entity relations under a unified schema. The dataset contains 13,000 manually annotated instances with 7 event types, 10 argument roles, 7 entity types, and 8 relation types. Experiments show that current supervised models, zero‑shot LLMs, and fine‑tuned LLMs struggle with relation extraction and exact span matching, highlighting the challenge of joint structured extraction in this domain.

By Yan Yu, Mengna Zhu, Zhenyu Song, Hao Yang, Haiwen Chen, Mao Wang
arXiv Computation and Language
Sep 14

Extracting Dataset Mentions in Forced Displacement and FCV Documents: A Weakly Supervised Framework with LLM-Based Label Refinement

The paper introduces a weakly supervised framework for extracting dataset mentions from forced displacement and Fragile, Conflict, and Violence (FCV) documents. It uses a lightweight model trained on general research literature to generate candidate mentions, which are then refined by a large language model that validates or rejects them and corrects boundaries. The refined annotations are augmented with synthetic and contrastive examples to fine‑tune the model, achieving 74.1% precision and 70.5% recall on a benchmark of 1,706 passages, with higher precision (89.5%) on passages that contain dataset references.

By Rafael Macalaba, Aivin V. Solatorio, Patrick Michael Brock, Olivier Dupriez
arXiv AI
Sep 10

Building evidence-based knowledge bases from full-text literature for disease-specific biomedical reasoning

EvidenceNet is a disease‑specific dataset that transforms full‑text biomedical literature into structured evidence records and graph representations, preserving study design, provenance, and quantitative support. Using an LLM‑assisted pipeline, it extracts experimentally grounded findings, normalizes entities, scores evidence quality, and links related records via typed semantic relations. The released subsets—EvidenceNet‑HCC and EvidenceNet‑CRC—contain thousands of evidence records and richly connected graphs, with high extraction and relation‑type accuracy, enabling retrieval‑augmented question answering and graph‑based tasks such as link prediction and target prioritization.

By Chang Zong, Jinyu Chen, Sicheng Lv, Si-tu Xue, Huilin Zheng, Jian Wan, Lei Zhang