← Back to all news
arXiv AI By Bohan Liu, Xiaohan Fan, Wenqian Ye, Aidong Zhang

Structure, Not Sequence: Tracking and Validating Folding Mechanisms in ESMFold's Trunk with Sparse Autoencoders

Read the original on arXiv AI →

The paper investigates how ESMFold’s folding trunk transforms amino‑acid sequences into 3D protein structures by using sparse autoencoders (SAEs) to uncover interpretable features. It first analyzes representation distributions across modules and recycles to identify distinct spatial computations, then shows that single representations carry inherited semantics while pair representations compute pairwise geometry. Finally, the authors use a reasoning language model and gradient‑based attribution to trace concept composition across blocks, verifying alignment with known biochemistry and demonstrating that the trained sparse dictionaries can predict the model’s fold quality, thereby turning the black‑box trunk into an interpretable, scientifically verifiable circuit.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Aug 28

Interpreting Latent Protein Language Model Features with Geometric Annotations

The paper introduces a scalable method to interpret sparse autoencoder (SAE) features in the ESM-2 protein language model by leveraging geometrically inspired features of the protein α‑carbon backbone. Across 8M layers of ESM-2, a false discovery rate–controlled analysis shows that local geometry is significantly associated with many SAE features, revealing substructure within known biological labels and enabling annotation of unannotated metagenomic proteins. Ablation experiments demonstrate that removing these geometric features shifts ESM-2’s predicted contact maps toward the descriptor, linking mechanistic interpretability with structural biology.

By Siddharth Setlur, Djordje Mihajlovic, Darrick Lee
llmsbenchmarkssafety
More like this →
arXiv AI
Sep 2

SymFold: Synergizing Evolutionary and Structural Priors for Accurate Protein Inverse Folding

SymFold introduces a symmetric dual‑path architecture that combines protein language models (PLMs) and multimodal protein language models (MPLMs) to iteratively guide protein sequence generation for inverse folding. By leveraging pretrained sequence evolution knowledge from PLMs and structural knowledge from MPLMs, the method improves upon the traditional serial pipeline where structure encoders produce coarse sequences refined by PLMs. Experiments on standard inverse‑folding benchmarks show state‑of‑the‑art performance, and ablation studies confirm the effectiveness of the symmetric design.

By Handong Wang, Jiaxin Qi, Baisheng Lai, Jianqiang Huang
llmsmultimodalbenchmarks
More like this →