arXiv AI

Advancing Clinical Trial Outcome Prediction with Domain-Specific Large Language Models

arXiv Computation and Language
Oct 9

Detecting Spin in Clinical Trials with Large Language Models

The paper presents a method for detecting spin in clinical trials by automatically identifying outcome switching using large language models (LLMs). The authors trained on 300 pairs of outcomes labeled for semantic similarity, then evaluated baseline text similarity models and open-source LLMs, applying prompt engineering, token‑probability classification, and majority voting. On a test set of 2,496 examples, the LLM‑based approach achieved an F1 score of 0.78 and accuracy of 0.90, surpassing baseline models but falling short of fine‑tuned BERT, and also produced natural‑language explanations that were manually evaluated for quality.

By Tja\v{s} Ajdovec, Marko Robnik-\v{S}ikonja, Simon \v{S}uster
arXiv Computation and Language
Oct 22

Is a 9B Decision Foundation Model Needed for Symptom-to-Disease Classification? Fine-Tuning Clef-Flash Against TF-IDF Baselines

The paper evaluates whether a 9‑billion parameter decision foundation model, Clef‑Flash, is necessary for symptom‑to‑disease classification. Fine‑tuning the model on the 24‑class Symptom2Disease benchmark yields 95.42% accuracy, but simple TF‑IDF linear classifiers achieve higher accuracy (98.75%) with far faster inference. Ablation studies show that predictions can hinge on single descriptive clauses, indicating that the benchmark may not fully capture the model’s capabilities.

By S. M. Mahbub Uddin Ahmed
arXiv Machine Learning
Jun 9

Do Larger Models Really Win in Drug Discovery? A Benchmark Assessment of Model Scaling in AI-Driven Molecular Property and Activity Prediction

arXiv:2604. 26498v3 Announce Type: replace Abstract: The rapid growth of molecular foundation models and large language models (LLMs) has encouraged a scale centred view of AI in drug discovery, in which larger pretrained models are expected to supersede compact cheminformatics models.

By Jinjiang Guo, Sheng Ding
arXiv AI
1d ago

Does Your Trial Simulator Know What the Drug Does? An Intervention-Blind Test of Interventional Validity on 7,207 Registered Trials

The study evaluates the interventional validity of clinical trial simulators by creating an intervention‑blind reference model that withholds intervention identifiers. Using 9,996 within‑trial contrasts from 7,207 phase 2/3 trials, the authors measure how much a simulator’s accuracy improves when it knows the intervention. Results show that a registry‑trained gradient‑boosting model performs poorly on interventional skill, while a large language model reading only the protocol achieves positive interventional skill, and perturbation tests confirm that the model relies on the intervention information.

By Alicem Koyun
arXiv AI
Oct 21

A general-purpose System One decision model for clinical records

The paper introduces Clinical System One (CS1), a 4‑billion‑parameter decision model that rapidly produces typed judgments from clinical records. Trained on 12.4 million Q&A pairs from 3.6 million patients, CS1 can answer multiple questions in a single forward pass, providing probability estimates for uncertain cases. In evaluations on 963,408 held‑out patients and 21 external datasets, CS1 outperformed larger open models and, when paired with a language‑model agent, efficiently identified trial‑eligible patients with far fewer tokens.

By Andrew Zhang, Trinh T. L. Vuong, Sophia J. Wagner, Dongmin Bang, Tong Ding, Yuchang Su, Mary Cipperman, Christina S. Chen, Judith Lefkes, Ramin Kahidi, Ming Y. Lu, Caiwei Tian, Tianhao Luo, Jingnan Ma, Rowland Pettit, Long Phi Le, Faisal Mahmood
arXiv AI
Oct 1

Defining and Categorising Human-AI Interactions in Clinical Trials: A Multidimensional Human-AI Classification Approach

The paper introduces a multidimensional framework for classifying human‑AI interactions (HAIIs) in clinical trials, categorising interactions by AI tasks, human‑AI relationships, interaction configurations, and interacting human groups. It evaluates the framework by applying it to 15 trials, with categorisation performed by both human reviewers and large language model classifiers. The study demonstrates that LLMs can assist in categorisation while highlighting the continued need for human judgment when trial records are incomplete or ambiguous.

By Sandra Woolley, Tim Collins, Khalid Khattak, Illia Chernomorets, Ariane Arevalo, Chris Richardson