arXiv Machine Learning
Do Vision CIL Methods Transfer to Protein? A Diagnostic Benchmark of Frozen Protein Encoders
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Oct 3
StabilityArc: Decoding Protein Sequence Embeddings into Generalizable Stability Landscapes
StabilityArc is a method that decodes protein sequence embeddings into generalizable stability landscapes. It uses a shared RoPE transformer to map frozen ESMC-600M residue representations into an Lx20 matrix of substitution effects, with a symmetric, contact-aware residual to predict epistasis. In extensive leave-one-protein-out tests on 134,794 ProteinGym variants, StabilityArc achieves a Spearman correlation of 0.7134, surpassing the best zero‑shot baseline, and further improves Kermut’s performance when used as a prior.
arXiv Machine Learning
Aug 28
Interpreting Latent Protein Language Model Features with Geometric Annotations
The paper introduces a scalable method to interpret sparse autoencoder (SAE) features in the ESM-2 protein language model by leveraging geometrically inspired features of the protein α‑carbon backbone. Across 8M layers of ESM-2, a false discovery rate–controlled analysis shows that local geometry is significantly associated with many SAE features, revealing substructure within known biological labels and enabling annotation of unannotated metagenomic proteins. Ablation experiments demonstrate that removing these geometric features shifts ESM-2’s predicted contact maps toward the descriptor, linking mechanistic interpretability with structural biology.
arXiv Machine Learning
Oct 15
ProtModernBERT: Compute-Efficient Protein Language Modeling
ProtModernBERT is a modernized BERT-style protein language model that reduces training cost by incorporating architectural refinements, achieving performance comparable to larger models while cutting pre‑training FLOPs by 80‑90%. The authors benchmark ProtModernBERT against ProtBert, Amplify, and ESM2 on supervised structural tasks and a zero‑shot functional benchmark, showing it matches or closely tracks Amplify‑120M and ProtBert. Their cluster‑based validation and random‑init experiments reveal that modern architectural components provide stronger inductive biases than parameter count in low‑compute regimes.
arXiv Computer Vision
Sep 14
DRDN: Decoupled Representation Dynamic Network for From-Scratch ViT Class-Incremental Learning
arXiv:2607.01630v2 Announce Type: replace Abstract: Dynamic expansion methods for class-incremental learning (CIL) protect task-specific knowledge by growing dedicated tokens or subnetworks, yet our...
arXiv Machine Learning
Aug 27
Interpreting Protein Language Model Embeddings via Orthogonal Projection for Protein Fitness Prediction
The paper introduces a method that uses orthogonal projection to remove the influence of known biochemical features from protein language model (PLM) embeddings, allowing the authors to assess how much these features contribute to protein fitness predictions. By applying this technique to high‑order and interaction effects, they demonstrate that eliminating these interpretable features reduces downstream classifier performance, indicating that PLM embeddings encode patterns correlated with biochemical properties. The authors also show that these biochemical features explain a substantial portion of the variance in the classifier’s predictions, suggesting that PLM embeddings capture biologically relevant information.
arXiv Machine Learning
Aug 13
Task- and dataset-specific information in protein language models
arXiv:2608. 12090v1 Announce Type: new Abstract: Protein language models (PLMs) have transferred the latest advances from natural language processing to computational biology.