arXiv Machine Learning
Jun 25
Two Stages of Folding: Convergent Mechanisms in AI Protein Folding Trunks
arXiv:2602. 06020v3 Announce Type: replace Abstract: How do protein structure prediction models fold proteins?
Structure, Not Sequence: Tracking and Validating Folding Mechanisms in ESMFold's Trunk with Sparse Autoencoders
The paper investigates how ESMFold’s folding trunk transforms amino‑acid sequences into 3D protein structures by using sparse autoencoders (SAEs) to uncover interpretable features. It first analyzes representation distributions across modules and recycles to identify distinct spatial computations, then shows that single representations carry inherited semantics while pair representations compute pairwise geometry. Finally, the authors use a reasoning language model and gradient‑based attribution to trace concept composition across blocks, verifying alignment with known biochemistry and demonstrating that the trained sparse dictionaries can predict the model’s fold quality, thereby turning the black‑box trunk into an interpretable, scientifically verifiable circuit.
Related stories
arXiv AI
Sep 1
Hyper-Fold: Exploring the Expressive Limit of Sequence-Geometry Learning for Proteins via Hypergraph Modeling
arXiv:2608.29207v1 Announce Type: new Abstract: Protein structure modeling rests on a single computational primitive: the interaction between what a residue is (sequence content) and where it sits (t...
arXiv Machine Learning
Aug 28
Interpreting Latent Protein Language Model Features with Geometric Annotations
The paper introduces a scalable method to interpret sparse autoencoder (SAE) features in the ESM-2 protein language model by leveraging geometrically inspired features of the protein α‑carbon backbone. Across 8M layers of ESM-2, a false discovery rate–controlled analysis shows that local geometry is significantly associated with many SAE features, revealing substructure within known biological labels and enabling annotation of unannotated metagenomic proteins. Ablation experiments demonstrate that removing these geometric features shifts ESM-2’s predicted contact maps toward the descriptor, linking mechanistic interpretability with structural biology.
arXiv AI
Sep 2
SymFold: Synergizing Evolutionary and Structural Priors for Accurate Protein Inverse Folding
SymFold introduces a symmetric dual‑path architecture that combines protein language models (PLMs) and multimodal protein language models (MPLMs) to iteratively guide protein sequence generation for inverse folding. By leveraging pretrained sequence evolution knowledge from PLMs and structural knowledge from MPLMs, the method improves upon the traditional serial pipeline where structure encoders produce coarse sequences refined by PLMs. Experiments on standard inverse‑folding benchmarks show state‑of‑the‑art performance, and ablation studies confirm the effectiveness of the symmetric design.
arXiv AI
Aug 18
Protein Structure Prediction: From Evolutionary Constraints to Generative Modeling
arXiv:2608. 16094v1 Announce Type: new Abstract: Accurate protein structure prediction is fundamental to structural biology because protein structure underlies molecular function and provides a basis for mechanistic interpretation.
arXiv Machine Learning
Jul 15
PRIME: Protein Representation via Physics-Informed Multiscale Equivariant Hierarchies
arXiv:2605. 01625v3 Announce Type: replace Abstract: Proteins are inherently multiscale physical systems whose functional properties emerge from coordinated structural organization across multiple spatial resolutions, ranging from atomic interactions to global fold topology.
arXiv Machine Learning
Jun 29
PairSAE: Mechanistic Interpretability from Pair Representations in Protein Co-Folding
arXiv:2606. 27440v1 Announce Type: new Abstract: Foundation models for structural biology have achieved remarkable performance in predicting biomolecular structure and show promise for the design of proteins and small molecules.
arXiv AI
Oct 1
Does Learning Protein Folding Generalize to Broader Reasoning?
arXiv:2609.38879v1 Announce Type: cross Abstract: Large language models rely heavily on human text, which often conveys surface answers rather than the spatial and structural logic behind them. Prote...
arXiv Machine Learning
Aug 6
RiboSphere: Learning Unified and Efficient Representations of RNA Structures
arXiv:2603. 19636v2 Announce Type: replace Abstract: Accurate RNA structure modeling remains difficult because RNA backbones are highly flexible, non-canonical interactions are prevalent, and experimentally determined 3D structures are comparatively scarce.
arXiv AI
Aug 12
Proteo-R1: Reasoning Foundation Models for De Novo Protein Design
arXiv:2605. 02937v2 Announce Type: replace-cross Abstract: Deep learning in de novo protein design has achieved atomic-level fidelity.
arXiv Machine Learning
Jul 20
Neural spectroscopy of AlphaFold2 reveals encoded protein conformational landscapes
arXiv:2607. 16087v1 Announce Type: new Abstract: AlphaFold2's 93 million parameters, shaped by the evolutionary record of protein structure encoded in the Protein Data Bank and in sequence alignments, are conventionally treated only as machinery for converting sequence to structure.
arXiv Machine Learning
Aug 21
ProteinZero: Self-Improving Protein Generation via Online Reinforcement Learning
arXiv:2506. 07459v4 Announce Type: replace Abstract: Protein generative models have shown remarkable promise in protein design, yet their success rates remain constrained by reliance on curated sequence-structure datasets and by misalignment between supervised objectives and real design goals.