arXiv Computation and Language

Custom Named Entity Recognition and Topic Classification for Global Health Publications

This thesis explores how to select and adapt NLP models for global health literature when annotated data and computational resources are scarce. It compares skip‑gram word2vec models trained on increasingly large specialized corpora with BioWordVec for semantic tag discovery, finding that larger coverage does not always yield more useful domain associations. The study also evaluates convolutional spaCy models versus a RoBERTa transformer for named entity recognition, noting a trade‑off between higher F1 scores and longer inference time, and investigates MiniLM few‑shot versus BART‑MNLI zero‑shot classification for multi‑label topic classification, highlighting practical constraints of inference cost. "whyItMatters":"The work provides empirical guidance on balancing model accuracy and resource demands for building knowledge systems in low‑resource global health settings."

arXiv Machine Learning
Sep 25

Language Specificity vs. Domain Diversity: Benchmarking Transformers for Bangla Medical NER

This study benchmarks transformer models for Bangla medical named entity recognition (NER), comparing BanglaBERT, multilingual BERT (mBERT), XLM‑RoBERTa, and GPT‑4o mini under zero‑shot and few‑shot prompting. Across a full test set of 3,179 samples, fine‑tuned XLM‑RoBERTa achieves a new state‑of‑the‑art F1‑score of 0.5959, while BanglaBERT lags with 0.4937, suggesting that domain diversity outweighs language specificity. The analysis shows high performance on Medicine and Specialist entities (F1 > 0.83) but lower accuracy on Symptoms (F1 0.4367), and demonstrates that fine‑tuned transformers outperform prompt‑only approaches by a factor of 3.76.

By Rakib Abdullah, Md. Maruful Islam Maruf
arXiv AI
1d ago

CardioLM: A multilingual suite of small language models for the cardiology domain

arXiv:2611.07940v1 Announce Type: cross Abstract: Background: Cardiovascular care generates large volumes of clinical free-text documentation in local languages. However, clinical natural language pr...

By Manuela-Daniela Danu, Miguel Rodr\'iguez-Ortega, Alberto Becerra-Tome, Pablo Arancibia, Fernando Gallego-Donoso, Lorenzo Pratesi, Xi Bai, Andrei-Damian Dinescu, Salvador Lima-L\'opez, Andrea Guala, Saskia Haitjema, Jonas Mack, Michele Miranda, Wesam Al Nabki, Jan Rodr\'iguez-Miret, Vasile George Marica, Vladim\'ir Kincl, Kate\v{r}ina Dost\'alov\'a, Michael Andrej, Alicia Ujil, Corinne Allaart, Machteld Boonstra, Malin Overmars, Anoop Shah, Giovanna Ceroni, Jonathan Kossoff, Domenico D'Amario, Renzo Laborante, Antonio Iaconelli, Isabella Fumarulo, Jacopo Lenkowicz, Livia Lilli, Emma Weiss, Alex Scafa, Felix Lindberg, Milou Gamage, Eul\`alia Farr\'e-Maduell, S\'ebastien Brati\`eres, Martin Krallinger, Bram van Es
arXiv Machine Learning
Jun 15

Automatic identification of diagnosis from hospital discharge letters via weakly supervised Natural Language Processing

arXiv:2410. 15051v3 Announce Type: replace-cross Abstract: Identifying patient diagnoses from hospital discharge letters is essential for large-scale cohort selection and epidemiological research, but traditional supervised approaches require extensive manual annotation, which is often impractical for large textual datasets.

By Vittorio Torri, Elisa Barbieri, Anna Cantarutti, Carlo Giaquinto, Francesca Ieva
arXiv AI
Sep 2

MiNER: Fine-Tuned Biomedical Natural Language Processing for Malaria Disease Entity Recognition in Clinical Texts

The paper introduces MiNER, a fine‑tuned biomedical NLP system that uses BioBERT to extract malaria‑related named entities from scientific literature. It builds a large, annotated corpus of malaria articles, preprocesses the text, and applies supervised learning to improve extraction performance. Experiments show that MiNER outperforms other encoding and machine‑learning methods in precision, recall, and accuracy, and the authors release the human‑labeled dataset for further research.

By V. S. Anoop, Devika N
arXiv Computation and Language
Sep 11

Cross-Lingual Clinical Annotation Projection as Constrained Text Generation: A Six-Language Study

The study investigates whether cross‑lingual clinical annotation projection can be treated as a constrained text‑generation task that preserves the original text while inserting entity tags. Using a workflow that embeds tags directly into immutable target‑language text and then validates them deterministically, the authors evaluated this approach against supervised candidate‑span projection and hybrid ML‑LLM refinement across six languages. Results show that direct LLM projection, particularly with GLM 5.2 and Gemma4:31B, achieves the highest strict F1 scores (up to 0.9201) and outperforms previous methods by 0.0564–0.1512, producing over 55,000 grounded mentions with accurate offsets.

By \'Alvaro Rey-Blanes, Francisco J. Moreno-Barea, Francisco J. Veredas
arXiv Computation and Language
Sep 4

Distilled Rapid Embedding Transfer (DRET): Parameter-Efficient Biomedical Domain Adaptation via Priority-Based Embedding Transfer

The paper introduces Distilled Rapid Embedding Transfer (DRET), a parameter‑efficient method that injects biomedical domain knowledge from large specialized models into a smaller general‑purpose model without retraining on the original specialized corpora. DRET evolves through iterative strategies—tokenizer‑merge (DRET 1.x), hybrid embedding averaging (DRET 2.0), priority‑based embedding transfer (DRET 3.x), and further refinements (DRET 4.x)—and demonstrates that a 66‑million‑parameter DistilBERT can achieve competitive or superior performance on token‑level PICO classification compared to much larger models, while remaining lightweight. The authors validate the embedding‑level transfer with cosine similarity, semantic‑shift, and t‑SNE analyses, highlighting DRET’s potential for scalable, resource‑efficient biomedical text mining.

By Girish Sundaram, Daniel Berleant
Hugging Face Trending Papers
Sep 10

Cross-Lingual Clinical Annotation Projection as Constrained Text Generation: A Six-Language Study

The study investigates whether cross‑lingual clinical annotation can be treated as a constrained text‑generation task that preserves the original text while inserting entity tags. Using a workflow that embeds tags directly into immutable target‑language text and then validates them deterministically, the authors compare this approach to supervised candidate‑span projection and hybrid ML‑LLM refinement across six languages. Results show that direct LLM projection, particularly with GLM 5.2 and Gemma4:31B, achieves the highest strict F1 scores, surpassing previous state‑of‑the‑art by up to 0.15 and producing over 55,000 grounded mentions with accurate offsets.

arXiv Computation and Language
Sep 16

SciNLP: A Domain-Specific Benchmark for Full-Text Scientific Entity and Relation Extraction in NLP

SciNLP is a new benchmark dataset for full‑text entity and relation extraction in the NLP domain, comprising 60 manually annotated papers with 6,429 entities and 1,649 relations. It is the first dataset to provide full‑text annotations of entities and their relationships specifically for NLP literature. Experiments show that models trained on SciNLP outperform baselines on certain tasks, and the dataset enabled the automatic construction of a fine‑grained knowledge graph with an average node degree of 3.3.

By Decheng Duan, Yingyi Zhang, Jitong Peng, Chengzhi Zhang
arXiv Machine Learning
Oct 9

BioBigBird: A Sparse Attention Model for Long-Range Dependency Processing in Biomedical Text

BioBigBird is a bidirectional language model tailored for biomedical text that uses sparse attention to process sequences up to 4096 tokens, addressing the limited context windows of existing domain-specific LLMs. It is pre‑trained on extensive biomedical literature and clinical data, with a multi‑stage training process to reduce noise and a multi‑task learning framework that jointly optimizes Named Entity Recognition and Relation Extraction. Evaluations on the BLURB benchmark show that the MTL‑enhanced BioBigBird achieves highly competitive results against state‑of‑the‑art models, and the models are publicly available on Hugging Face.

By Roshan Balaji, Pavan Kumar S, Vasudev Gupta, Sreejith N, Keerthana Sridhar, Nirav Bhatt