arXiv Computation and Language By Girish Sundaram, Daniel Berleant

Distilled Rapid Embedding Transfer (DRET): Parameter-Efficient Biomedical Domain Adaptation via Priority-Based Embedding Transfer

Read the original on arXiv Computation and Language →

The paper introduces Distilled Rapid Embedding Transfer (DRET), a parameter‑efficient method that injects biomedical domain knowledge from large specialized models into a smaller general‑purpose model without retraining on the original specialized corpora. DRET evolves through iterative strategies—tokenizer‑merge (DRET 1.x), hybrid embedding averaging (DRET 2.0), priority‑based embedding transfer (DRET 3.x), and further refinements (DRET 4.x)—and demonstrates that a 66‑million‑parameter DistilBERT can achieve competitive or superior performance on token‑level PICO classification compared to much larger models, while remaining lightweight. The authors validate the embedding‑level transfer with cosine similarity, semantic‑shift, and t‑SNE analyses, highlighting DRET’s potential for scalable, resource‑efficient biomedical text mining.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Computation and Language
Sep 22

Custom Named Entity Recognition and Topic Classification for Global Health Publications

This thesis explores how to select and adapt NLP models for global health literature when annotated data and computational resources are scarce. It compares skip‑gram word2vec models trained on increasingly large specialized corpora with BioWordVec for semantic tag discovery, finding that larger coverage does not always yield more useful domain associations. The study also evaluates convolutional spaCy models versus a RoBERTa transformer for named entity recognition, noting a trade‑off between higher F1 scores and longer inference time, and investigates MiniLM few‑shot versus BART‑MNLI zero‑shot classification for multi‑label topic classification, highlighting practical constraints of inference cost. "whyItMatters":"The work provides empirical guidance on balancing model accuracy and resource demands for building knowledge systems in low‑resource global health settings."

By Genis Skura, Antoine Geissb\"uhler, Jean-Luc Falcone
arXiv AI
Oct 23

EHRAdapt: Adapting Pretrained Language Models to Electronic Health Records with Semantic Priors for Rare Clinical Events

EHRAdapt is an adapter that maps electronic health record tuples directly into a frozen language model’s embedding space, using learned embeddings for modality, time gaps, and dedicated vectors for event codes. It represents each event vector as a sum of a frozen, ontology‑trained semantic prior and a low‑rank evidence residual, enabling robust learning even for rare clinical events. Experiments on four million patient records with three large language model backbones show that the full adapter outperforms all ablations in next‑event prediction and improves rare‑disease discrimination in classification tasks.

By Andre R Goncalves, Vincent Liu, Priyadip Ray
arXiv Computation and Language
Sep 23

BELXTR: Biomedical Entity Linking via Contextualized Token Retrieval

BELXTR is a new biomedical entity linking model that uses a multi‑vector (late interaction) architecture to preserve token‑level matching information, unlike traditional embedding‑based approaches that compress mentions into a single vector. By extending the XTR model with a task‑specific training objective and active query expansion, BELXTR achieves state‑of‑the‑art performance on half of ten evaluated corpora, with an average 5‑percentage‑point gain in recall@1. The model shows especially strong results on cross‑species gene disambiguation, outperforming an LLM‑powered retrieve‑and‑rerank pipeline and approaching a specialized rule‑based system.

By Samuele Garda, Ulf Leser
arXiv Computation and Language
Oct 22

Overview of CHIP 2026 Shared Task 1: Patient-Specific Phenotype Concept Recognition in Full-Text Biomedical Literature

The paper introduces CHIP 2026 Shared Task 1, which focuses on patient‑specific phenotype concept recognition in full‑text biomedical literature. The task is split into two subtasks—full‑text phenotype concept recognition and patient‑specific phenotype association—using 200 annotated articles from the PatientPheX dataset that include 506 patients, 15,193 phenotype mentions, and 5,656 patient‑phenotype associations. A large community of 455 teams participated, with the best system scoring 0.7389, surpassing the baseline by 8.37 percentage points, and demonstrating the effectiveness of biomedical language models, large language models, and ensemble strategies while highlighting ongoing challenges in patient attribution and phenotype normalization.

By Xuanlyu Zhu, Feng Tian, Yang Zhong, Pengqi Zheng, Xiwen Bi, Ling Luo, Yuanyuan Sun, Jian Wang, Lingfei Lin