ProtLingo: Efficient Protein Language Modeling via Conditional Memory and Expert Routing
Read the original on arXiv AI →ProtLingo is a protein language modeling framework that enhances a pretrained single‑sequence Transformer backbone with conditional local memory and sparse expert routing. It maps residue representations into discrete codes, composes local windows into latent N‑gram addresses, and retrieves reusable residual signals for recurring sequence contexts. The model also converts selected feed‑forward blocks into sparse Mixture‑of‑Experts layers, allowing residue‑dependent computation while activating only a subset of parameters, achieving competitive performance on protein fitness prediction, FLIP benchmarks, and supervised contact prediction with a 150M‑parameter backbone.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.