arXiv:2607. 19426v1 Announce Type: cross Abstract: Single-cell datasets are increasingly costly to store, audit, and reuse for model training.
By Yaodi Luo, Peize He, Bowen Han, Lingbei Mengg
arXiv:2608.00152v3 Announce Type: replace-cross
Abstract: AI evaluation can support the wrong inference when an in-domain benchmark success does not survive distribution shift, or when the benchmark...
By Mehrdad Shoeibi, Niloofar Yousefi
The paper investigates when post‑training quantization (PTQ) calibration helps or hurts text embedders, focusing on the amount of calibration (dose) and whether to calibrate at all. Across three models and 161 calibration runs at 4‑bit weight quantization, the effect ranges from significant gains to catastrophic degradation, driven mainly by the functional displacement of the quantized model. A validation‑gated abstention rule is proposed that avoids harmful calibrations while capturing most benefits, highlighting the trade‑off between safety and opportunity.
By Hyojung Han, Ryangkyung Kang, Youngeun Kim, Seung-Hun Jeon
The study evaluates a frozen Geneformer representation for predicting CRISPRi perturbation effects under a tightly controlled, pre‑registered protocol. While the representation shows significant predictive power within the Virtual Cell Challenge dataset, it fails to transfer to external screens, with negative zero‑shot Spearman correlations. The analysis also reveals that the VCC endpoint is heavily influenced by sampling depth, as cell count alone explains most of the variance, indicating a sampling‑depth entanglement that could mask transfer failures in less controlled settings.
By Mehrdad Shoeibi, Niloofar Yousefi
arXiv:2607. 19426v2 Announce Type: replace-cross Abstract: Large single-cell datasets are expensive to store, curate, and repeatedly reuse for model training.
By Yaodi Luo, Peize He, Lingbei Meng, Bowen Han, Zheng Lu, Jianqing Zhu, Lian Zhang
PRISM is a tile‑level benchmark for pathology foundation models that evaluates reliability across four dimensions: model choice, label fraction, calibration, and out‑of‑distribution transfer. It tests eight models on six datasets at six label fractions (1%–100%) with 864 linear probe runs, plus a five‑hospital Camelyon17 transfer study. The benchmark reveals four key phenomena, including the changing relationship between AUROC and calibration at low labels, collapse of linear probes on harder datasets, the impact of source‑domain labels on target calibration, and limitations of temperature scaling.
By Alicem Koyun