BioBigBird is a bidirectional language model tailored for biomedical text that uses sparse attention to process sequences up to 4096 tokens, addressing the limited context windows of existing domain-specific LLMs. It is pre‑trained on extensive biomedical literature and clinical data, with a multi‑stage training process to reduce noise and a multi‑task learning framework that jointly optimizes Named Entity Recognition and Relation Extraction. Evaluations on the BLURB benchmark show that the MTL‑enhanced BioBigBird achieves highly competitive results against state‑of‑the‑art models, and the models are publicly available on Hugging Face.
By Roshan Balaji, Pavan Kumar S, Vasudev Gupta, Sreejith N, Keerthana Sridhar, Nirav Bhatt
The paper introduces pre‑trained models for extracting variant‑phenotype relations from biomedical text, focusing on the SNPPhenA corpus. Fine‑tuning small BERT‑based models, especially DeBERTa, achieves performance close to the current state‑of‑the‑art. Moreover, careful fine‑tuning of Google’s Gemini Pro 1.0 surpasses existing benchmarks on both sentence‑level and abstract‑level relation extraction tasks.
By Claudiu Creanga, Liviu P. Dinu, Daniela Gifu
This thesis explores how to select and adapt NLP models for global health literature when annotated data and computational resources are scarce. It compares skip‑gram word2vec models trained on increasingly large specialized corpora with BioWordVec for semantic tag discovery, finding that larger coverage does not always yield more useful domain associations. The study also evaluates convolutional spaCy models versus a RoBERTa transformer for named entity recognition, noting a trade‑off between higher F1 scores and longer inference time, and investigates MiniLM few‑shot versus BART‑MNLI zero‑shot classification for multi‑label topic classification, highlighting practical constraints of inference cost.
"whyItMatters":"The work provides empirical guidance on balancing model accuracy and resource demands for building knowledge systems in low‑resource global health settings."
By Genis Skura, Antoine Geissb\"uhler, Jean-Luc Falcone
arXiv:2608. 19201v1 Announce Type: cross Abstract: Bioinformatics software and databases are essential components of modern life science research, yet their mentions in the scientific literature are often inconsistent and difficult to systematically identify at scale.
By Hao Xuan, Rithvij Pasupuleti, Ben Liu, Haishuo Sun, Jun Zhang, Zijun Yao, Cuncong Zhong
arXiv:2606. 13051v1 Announce Type: new Abstract: Despite advances in information extraction driven by deep learning and large language models, performance gaps remain in highly specialized biomedical fields, where domainspecific complexity poses challenges for generalist models.
By Fabien Maury (Imagine - U1163, HeKA | U1346), Sol\`ene Grosdidier (Imagine - U1163), Maud de Dieuleveult (Imagine - U1163), Adrien Coulet (HeKA | U1346)
The paper introduces a configurable semantic chunking framework for biomedical information extraction in retrieval‑augmented generation systems. It replaces the fixed‑size chunking stage of BioMedRAG with entity‑preserving windows, trigger‑centered chunking, proposition‑first extraction, tiered trigger prioritization, and hierarchical relation resolution, while keeping the rest of the pipeline unchanged. Experiments on relation extraction benchmarks (GM‑CIHT, DDI, ChemProt) and adverse event classification (ADE) show that the hybrid configuration boosts performance on datasets with explicit relation cues, achieving 82.6% F1 on GM‑CIHT compared to 74.2% with the baseline.
By Riya Ahuja (Institute of Data Science in Biomedicine, TU Braunschweig, Braunschweig, Germany, Braunschweig Integrated Centre of Systems Biology, TU Braunschweig, Braunschweig, Germany), Tim Kacprowski (Institute of Data Science in Biomedicine, TU Braunschweig, Braunschweig, Germany, Braunschweig Integrated Centre of Systems Biology, TU Braunschweig, Braunschweig, Germany), Roya Shiasi Sardoabi (Institute of Data Science in Biomedicine, TU Braunschweig, Braunschweig, Germany, Braunschweig Integrated Centre of Systems Biology, TU Braunschweig, Braunschweig, Germany)