arXiv Machine Learning

GeneICL: A Tabular Foundation Model for Bulk Transcriptomics

arXiv AI
3d ago

Transcriptome-informed multi-modal AI for predicting neoadjuvant therapy response from breast cancer biopsies

arXiv:2610.03693v1 Announce Type: new Abstract: Scarcity of labeled data limits development of deep learning biomarkers in oncology. We develop a two-stage AI model predicting pathological complete r...

By Jungkyu Park, Dhruva Biswas, Joseph Cappadona, Cerise Tang, Ken G. Zeng, Bartosz Machura, Chuwen Liu, Paolo Tarantino, Coral Omene, Francisco J. Esteva, Rohit Bhargava, Marcin Braun, Kamila Pa\'zdzierz, Jakub Czerwi\'nski, Hanna Roma\'nska-Knight, Albert Grinshpun, Bareket Daniel, Michele Buchinger, Frederick Howard, Piotr Wysocki, Brie Chun, Freya Schnabel, Rich Caruana, Jan Witowski, Krzysztof J. Geras
arXiv Machine Learning
Jul 21

Tabular Foundation Models Can Do Survival Analysis

arXiv:2601. 22259v2 Announce Type: replace Abstract: While tabular foundation models have achieved remarkable success in classification and regression, adapting them to model time-to-event outcomes for survival analysis is non-trivial due to right-censoring, where data observations may end before the event of interest occurs.

By Da In Kim, Wei Siang Lai, Kelly W. Zhang
arXiv AI
Oct 1

CellMSA: Context Modeling for Single-Cell Representation Learning

CellMSA introduces a novel single‑cell representation learning framework that leverages a multiple‑sequence‑alignment‑inspired context model. For each target cell, it retrieves relevant cells across batches and related cell types, summarizing cross‑cell patterns into a context‑dependent gene‑pair representation that is fed into a pair‑aware encoder. Pretraining on a massive human single‑cell corpus (≈109 million cells) and subsequent benchmarks demonstrate consistent performance gains over existing methods.

By Suyuan Zhao, Minghao Liu, Yizhen Luo, Zaiqing Nie
arXiv Machine Learning
Jun 4

SurvPFN: Towards Foundation Models for Survival Predictions

arXiv:2606. 04564v1 Announce Type: new Abstract: Tabular foundation models (TFMs) have made rapid progress in standard classification and regression, but time-to-event survival prediction tasks have remained largely untouched.

By Samuel B\"ohm (Institute of Epidemiology and Prevention, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany), Lennart Purucker (Department of Computer Science, University of Freiburg, Freiburg, Germany, PriorLabs, Freiburg, Germany), Frank Hutter (Department of Computer Science, University of Freiburg, Freiburg, Germany, PriorLabs, Freiburg, Germany), Pascal Schlosser (Institute of Epidemiology and Prevention, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany, Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, US, CIBSS - Centre for Integrative Biological Signalling Studies, University of Freiburg, Freiburg, Germany)
arXiv Machine Learning
Sep 17

TabICLv2: A better, faster, scalable, and open tabular foundation model

TabICLv2 is a new state‑of‑the‑art tabular foundation model that outperforms existing methods on regression and classification tasks. It relies on a synthetic data generation engine for diverse pretraining, architectural innovations such as a scalable softmax attention, and optimized training protocols that replace AdamW with the Muon optimizer. On the TabArena and TALENT benchmarks, TabICLv2 surpasses the current best model, RealTabPFN‑2.5, without any tuning, while also being faster and capable of handling million‑scale datasets with limited GPU memory.

By Jingang Qu, David Holzm\"uller, Ga\"el Varoquaux, Marine Le Morvan
arXiv AI
Sep 7

Adaptation Interfaces for In-Context Tabular Foundation Models in Time-to-Event Prediction

The paper explores how to adapt tabular foundation models (TabFMs) for censored time‑to‑event prediction by linking them with CoxPH and DeepHit and revising training procedures. It evaluates zero‑shot, classification‑based fine‑tuning, and survival‑head adaptations across 74 single‑risk and 4 competing‑risk datasets, finding that zero‑shot works best on small datasets while supervised adaptation excels as data grows. The study shows that the choice of adaptation interface and data regime critically influences TabFM transfer performance.

By Minh-Khoi Pham, Luca Cotugno, Dan Cernei, Alina Sirbu, Stefano Masi, Giuseppe Prencipe, Alessandro Pingitore, Patrizia Landi, Working Group on Uric Acid, Cardiovascular Risk of the Italian Society of Hypertension, Tai Tan Mai, Martin Crane, Marija Bezbradica
arXiv Machine Learning
Jun 2

OncoReason: Structuring Clinical Reasoning in LLMs for Robust and Interpretable Survival Prediction

arXiv:2510. 17532v2 Announce Type: replace-cross Abstract: Predicting cancer treatment outcomes requires models that are both accurate and interpretable, particularly in the presence of heterogeneous clinical data.

By Raghu Vamshi Hemadri, Geetha Krishna Guruju, Kristi Topollai, Anna Ewa Choromanska
arXiv AI
Sep 3

General Demographic Pre-trained Models for Enhancing Predictive Performance Across Diseases and Population

The paper introduces the General Demographic Pre-trained (GDP) model, a lightweight foundation model that learns representations from the two most common clinical attributes—age and sex. By optimizing encoding and visit‑reordering strategies, GDP embeddings are shown to improve predictive performance when concatenated with raw features across various disease and geographic cohorts. The model outperforms several state‑of‑the‑art tabular foundation models and tree‑based algorithms, demonstrating that enriched demographic embeddings can enhance classification tasks while remaining fully compatible with standard classifiers.

By Li-Chin Chen, Ji-Tian Sheu, Yuh-Jue Chuang