arXiv Machine Learning June 9, 2026 By Ihor Kendiukhov

Scaling Laws for Masked-Reconstruction Transformers on Single-Cell Transcriptomics

Read the original on arXiv Machine Learning →

arXiv:2602. 15253v2 Announce Type: replace Abstract: Neural scaling laws -- power-law relationships between loss, model size, and data -- have been extensively documented for language and vision transformers, yet their existence in single-cell genomics remains largely unexplored.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Aug 5

Scaling an Autoregressive Transformer for Single-Cell Generation

arXiv:2608. 02961v1 Announce Type: cross Abstract: We study a self-supervised generation task for single-cell gene expression vectors: given a set of vectors from a cell type, we aim to generate additional gene expression vectors of that cell type.

By Aleksandr Sharipov, Yusif Mukhtarov, Igor Molybog

llms nlp efficiency

arXiv Machine Learning

Jun 9

Integrating gene regulatory priors into Transformer attention with scTransformer for interpretable scRNA-seq analysis

arXiv:2606. 09558v1 Announce Type: cross Abstract: Motivation: Transformer-based models are increasingly applied to large-scale single-cell transcriptomics, showing strong performance through self-supervised learning on millions of cells.

By Mikele Milia, Louis Fabrice Tshimanga, Henning Mueller, Manfredo Atzori, Barbara Di Camillo

llms rag safety

arXiv Machine Learning

Aug 4

Beyond Gene Reconstruction: Learning Cell Representations through Complementary Transcriptomic Views

arXiv:2608. 00985v1 Announce Type: new Abstract: The rapid growth of single-cell transcriptomic data has enabled the development of foundation models pretrained primarily by reconstructing masked expression values.

By Jiaqi Xiong, Yuntao hu, Yu Zheng, Yifei Shi, Xinyue Guo, Jiaxin Qi

arXiv Machine Learning

Jul 17

A vision foundation model for single-cell biology via spatial gene cartography

arXiv:2607. 14163v1 Announce Type: cross Abstract: Most single-cell foundation models are adapted from language models, representing each cell as a sequence of gene tokens.

By Ridvan Yesiloglu, Sakib Mostafa, James Zou, Ash Alizadeh, Jiajun Wu, Lei Xing, Ehsan Adeli, Md Tauhidul Islam

llms computer-vision fine-tuning

arXiv Machine Learning

Jun 9

scCBGM: Interpretable Single-Cell Counterfactual Editing

arXiv:2606. 07760v1 Announce Type: new Abstract: Understanding cellular phenotypes and how they respond to perturbations is critical for disease biology and therapeutic design.

By Alma Andersson, Aya Abdelsalam Ismail, Edward De Brouwer, Doron Haviv, Tommaso Biancalani, Kyunghyun Cho, Gabriele Scalia, A\"icha BenTaieb, Hector Corrada Bravo

diffusion benchmarks

arXiv AI

Aug 10

CellWorld: From Gene-Level Reconstruction to Latent Cell Prediction in Spatial Transcriptomics Foundation Models

arXiv:2608. 06659v1 Announce Type: new Abstract: This paper shows that latent-space predictive pretraining can provide a scalable route to foundation models for spatial transcriptomics.

By Haiping Liu, Qian Zhao, Lijing Lin, Jingyuan Sun, Hongpeng Zhou

fine-tuning benchmarks

Related stories

Scaling an Autoregressive Transformer for Single-Cell Generation

Integrating gene regulatory priors into Transformer attention with scTransformer for interpretable scRNA-seq analysis

Beyond Gene Reconstruction: Learning Cell Representations through Complementary Transcriptomic Views

A vision foundation model for single-cell biology via spatial gene cartography

scCBGM: Interpretable Single-Cell Counterfactual Editing

CellWorld: From Gene-Level Reconstruction to Latent Cell Prediction in Spatial Transcriptomics Foundation Models