The paper presents a 3D foundation model for light sheet fluorescence microscopy (LSM) that is pretrained on a large curated set of 3D images from various organisms, stains, and imaging protocols. By jointly optimizing for masked reconstruction and image‑text alignment, the model learns transferable volumetric representations that dramatically reduce the need for annotated data. The pretrained backbone enables efficient few‑shot adaptation to downstream tasks such as segmentation, classification, and deblurring, consistently outperforming baselines according to standard metrics and expert evaluation.
By Adina Scheinfeld, Haotan Zhang, Shang Mu, Rudolf L. M. van Herten, Lucas Stoffl, Ali Erturk, Zhuhao Wu, Johannes C. Paetzold
Atelier is a self‑supervised framework that uses a transformer‑based hypernetwork to generate implicit neural representations (INRs) for cryo‑EM maps, enabling efficient, scale‑agnostic, coordinate‑conditioned feature extraction. Trained on 5,439 maps from the Electron Microscopy Data Bank, the pretrained INR provides continuous local feature fields that can be used as auxiliary channels for a 3D nested U‑Net, improving voxel‑level property prediction across eight tasks compared to a volume‑only baseline. The approach demonstrates that amortized INRs can serve as a geometry‑aware primitive for large‑scale cryo‑EM analysis.
By Phillip Lo, Sudarshan Babu, Dari Kimanius, Aly A. Khan
arXiv:2606. 03888v1 Announce Type: cross Abstract: Self-supervised learning has enabled large-scale pre-training on 2D natural images, producing general-purpose visual representations that transfer effectively across tasks.
By Ioannis Gatopoulos, Nicolas K\"anzig, Sebastian Ot\'alora, Fei Tang
The study evaluates self‑supervised learning (SSL) models pretrained on ImageNet‑1k and the Human Protein Atlas (HPA) Field‑of‑View (FOV) for protein localization in microscopy images. DINO‑based Vision Transformer backbones pretrained on either dataset transfer well to the OpenCell dataset, achieving strong performance even without fine‑tuning and improving further when fine‑tuned (0.704 ± 0.027 macro F1 on 17 classes). At the single‑cell level, the HPA‑pretrained model outperforms others in k‑nearest‑neighbor classification across all neighborhood sizes (macro F1 ≥ 0.515).
By Ben Isselmann, Dilara G\"oksu, Heinz Neumann, Andreas Weinmann
arXiv:2607. 17782v1 Announce Type: cross Abstract: Foundation models pretrained using self-supervised learning have transformed computer vision by learning transferable representations from large-scale unlabeled data.
By Moona Mazher, Abdul Qayyum, Steven A. Niederer, Daniel C. Alexander
arXiv:2607. 22712v1 Announce Type: cross Abstract: Single-cell light microscopy images have become an important data source for characterizing cell phenotypes, but their complexity and heterogeneity pose challenges to high-throughput automated analysis.
By Yifan Shang (Department of Biomedical Engineering, The Chinese University of Hong Kong, Hong Kong, China, College of Computer Science and Electronic Engineering, Hunan University, Changsha, China), Jiahui Tan (College of Computer Science and Electronic Engineering, Hunan University, Changsha, China), Xiangxiang Zeng (College of Computer Science and Electronic Engineering, Hunan University, Changsha, China), Renjie Zhou (Department of Biomedical Engineering, The Chinese University of Hong Kong, Hong Kong, China)
The paper introduces a scalable method to interpret sparse autoencoder (SAE) features in the ESM-2 protein language model by leveraging geometrically inspired features of the protein α‑carbon backbone. Across 8M layers of ESM-2, a false discovery rate–controlled analysis shows that local geometry is significantly associated with many SAE features, revealing substructure within known biological labels and enabling annotation of unannotated metagenomic proteins. Ablation experiments demonstrate that removing these geometric features shifts ESM-2’s predicted contact maps toward the descriptor, linking mechanistic interpretability with structural biology.
By Siddharth Setlur, Djordje Mihajlovic, Darrick Lee
arXiv:2609.23019v1 Announce Type: cross
Abstract: Soma instance segmentation, i.e., identifying and delineating individual cell somas as distinct instances, is crucial for cellular analysis and conne...
By Mohammad Khateri, Morteza Ghahremani, Jussi Tohka, Alejandra Sierra
arXiv:2603. 13377v2 Announce Type: replace-cross Abstract: Representation learning has driven major advances in natural image analysis by enabling models to acquire high-level semantic features.
By Ivan Svatko, Maxime Sanchez, Ihab Bendidi, Gilles Cottrell, Auguste Genovesio
arXiv:2608. 07575v1 Announce Type: cross Abstract: Confocal microscopy of optically cleared and swelled tissue resolves complex biological structures in 3D, but such acquisitions are highly anisotropic: along the under-sampled axial direction the structure can appear discontinuous, hampering reconstruction and automated quantitative analysis.
By Arash Fatehi, Robin Ebbestad, Linus Butt, Hans Blom, Sigrid Lundberg, Hannes Olauson, Hjalmar Brismar, David Unnersj\"o-Jess, Thomas Benzing, Katarzyna Bozek
arXiv:2604.19609v2 Announce Type: replace
Abstract: Transformers have become a common foundation across deep learning, yet 3D scene understanding still relies on specialized backbones with strong dom...
By Kadir Yilmaz, Adrian Kruse, Tristan H\"ofer, Daan de Geus, Bastian Leibe
The paper introduces CARNIVAL, a model for protein annotation in cryo-electron tomography (cryo-ET) volumes that leverages simulated data and a forward model to generate domain‑specific augmented paired views for self‑supervised training. By incorporating simulation‑derived protein positions and identities into the architecture and loss function, the model localises semantic information at protein locations. CARNIVAL is evaluated on real tomograms without finetuning and outperforms a state‑of‑the‑art contrastive model that lacks forward‑model paired views or privileged information.
By Bogdan Toader, Kiarash Jamali, Tanmay A. M. Bharat, Sjors H. W. Scheres