← Back to all news
arXiv Computation and Language By Rami Luisto, Liisa Pet\"ainen, Tommi Gr\"onholm, Jan B\"ohm, Maarit Ahtiainen, Tomi Lilja, Ilkka P\"ol\"onen, Sami \"Ayr\"am\"o

Continued Pretraining of FinBERT on Finnish Histopathological Reports: Train-Time Signals and Proxy Downstream Correlations

Read the original on arXiv Computation and Language →

The paper investigates continued pretraining (CPT) of the Finnish BERT model (FinBERT) on a Finnish histopathological dataset, noting that CPT train‑time loss curves vary significantly across domains. Since the histopathology data lacks labels, the authors use public Finnish datasets as proxy data to examine whether CPT‑derived signals correlate with downstream classification gains. Their exploratory analysis finds that certain CPT features are associated with improved proxy classification performance, adding to the sparse literature on Finnish healthcare NLP.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv AI
Aug 28

Making Clinical Language Models Auditable: Concept-Guided Fine-Tuning for Robust Prediction

The paper introduces CAST, a concept-guided artifact suppression tuning framework that uses sparse autoencoders to identify and suppress note-specific artifacts in clinical language models. CAST labels latent features with an LLM-assisted pipeline and ICD‑10 constraints, then fine‑tunes the model while providing post‑hoc per‑concept attributions for auditability. In experiments on MIMIC‑IV discharge‑note mortality prediction, CAST outperforms standard fine‑tuned encoders and competes with strong LLM baselines while offering a feature‑level audit trail of clinical concepts and suppressed artifacts.

By Jin Mu, Guanhua Chen
llmsnlpfine-tuning
More like this →
arXiv Machine Learning
Sep 10

Where Does the Signal Live? A Web Data Recipe for Medical Encoder Pretraining

The paper introduces a web‑data curation recipe for pretraining medical encoders, addressing the scarcity of large, diverse corpora in dense‑terminology domains like medicine. It proposes two complementary techniques: medical‑term density filtering to select documents rich in medical terminology, and signal‑amplifying rephrasing that uses an LLM to rewrite documents into denser variants with broader entity contexts. Applied to French medical NLP, the recipe produces the FineMed corpus and the DoctoBERT encoder family, achieving state‑of‑the‑art results on the DrBenchmark public benchmark and a proprietary clinical NER task.

By Bofeng Huang, Jacques Sun, Diane Bouchacourt, Nicolas Barascud, Fajwel Fogel
llmsnlpbenchmarks
More like this →
arXiv Machine Learning
Jul 20

LLM4EHR: Aligning Clinical Time Series with Medical Event Sequences via Large Language Models

arXiv:2607. 15447v1 Announce Type: new Abstract: Recent research in clinical machine learning, focusing on outcome predictions in intensive care unit (ICU), has shifted from bespoke supervised models to foundation models, utilising modern representation learning methods.

By Jingteng Li, Alexander Capstick, Louise Rigny, Iona Biggart, Neil J Sebire, Payam Barnaghi
llmsragbenchmarks
More like this →
arXiv Machine Learning
Jun 15

Automatic identification of diagnosis from hospital discharge letters via weakly supervised Natural Language Processing

arXiv:2410. 15051v3 Announce Type: replace-cross Abstract: Identifying patient diagnoses from hospital discharge letters is essential for large-scale cohort selection and epidemiological research, but traditional supervised approaches require extensive manual annotation, which is often impractical for large textual datasets.

By Vittorio Torri, Elisa Barbieri, Anna Cantarutti, Carlo Giaquinto, Francesca Ieva
llmsragnlp
More like this →