← Back to all news
arXiv Machine Learning By Zeyu Dong, Jiahui Zhong

Rethinking Class Imbalance for Single-Cell Foundation Models: A Systematic Benchmark Across Architectures and Long-Tail Loss Functions

Read the original on arXiv Machine Learning →

The paper benchmarks six long‑tail loss functions—cross‑entropy, weighted CE, class‑balanced loss, focal loss, LDAM, and logit‑adjusted softmax—across three single‑cell foundation model architectures (scGPT, scBERT, Geneformer) and three datasets (Multiple Sclerosis, Zheng68K, human Pancreas). It shows that overall accuracy masks systematic failures on rare, disease‑relevant cell types, with a consistent gap between overall accuracy, Macro‑F1, and rare‑class recall under plain cross‑entropy. The study identifies two distinct regimes of rare‑class failure, predicts reweighting efficacy by absolute training‑set size, and finds class‑balanced loss and LDAM to be the most reliable across all settings.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Jul 14

A Strong Balanced-Softmax Classifier-Retraining Baseline for Long-Tailed Recognition

arXiv:2607. 09832v1 Announce Type: new Abstract: Long-tailed recognition methods often modify losses, margins, or representations to reduce the dominance of frequent classes.

By Juan Terven, Diana Margarita C\'ordova Esparza, Julio Alejandro Romero Gonzalez, Edgar Arturo Ch\'avez Urbiola, Francisco Javier Willars Rodriguez, Juan Bautista Hurtado Ramos, Alfonso Ramirez Pedraza
More like this →
arXiv Computation and Language
Sep 4

Distilled Rapid Embedding Transfer (DRET): Parameter-Efficient Biomedical Domain Adaptation via Priority-Based Embedding Transfer

The paper introduces Distilled Rapid Embedding Transfer (DRET), a parameter‑efficient method that injects biomedical domain knowledge from large specialized models into a smaller general‑purpose model without retraining on the original specialized corpora. DRET evolves through iterative strategies—tokenizer‑merge (DRET 1.x), hybrid embedding averaging (DRET 2.0), priority‑based embedding transfer (DRET 3.x), and further refinements (DRET 4.x)—and demonstrates that a 66‑million‑parameter DistilBERT can achieve competitive or superior performance on token‑level PICO classification compared to much larger models, while remaining lightweight. The authors validate the embedding‑level transfer with cosine similarity, semantic‑shift, and t‑SNE analyses, highlighting DRET’s potential for scalable, resource‑efficient biomedical text mining.

By Girish Sundaram, Daniel Berleant
llmsragnlpfine-tuning
More like this →