Accelerating Protein Language Model ProtST on Intel Gaudi 2

Fast Inference on Large Language Models: BLOOMZ on Habana Gaudi2 Accelerator

ProtLingo: Efficient Protein Language Modeling via Conditional Memory and Expert Routing

ProtLingo is a protein language modeling framework that enhances a pretrained single‑sequence Transformer backbone with conditional local memory and sparse expert routing. It maps residue representations into discrete codes, composes local windows into latent N‑gram addresses, and retrieves reusable residual signals for recurring sequence contexts. The model also converts selected feed‑forward blocks into sparse Mixture‑of‑Experts layers, allowing residue‑dependent computation while activating only a subset of parameters, achieving competitive performance on protein fitness prediction, FLIP benchmarks, and supervised contact prediction with a 150M‑parameter backbone.

LEMON-ZEST: Evolution-Informed Tokenization for Efficient Protein Language Modeling

arXiv:2609.37675v1 Announce Type: new Abstract: Protein Language Models (PLMs) have made remarkable progress following scaling laws established in natural language processing across sequence- and str...

Accelerating Vision-Language Models: BridgeTower on Habana Gaudi2

Training mRNA Language Models Across 25 Species for $165

Analysis of Quantized and Efficiently Adapted Protein Language Models

The study evaluates 4‑bit quantization and low‑rank adapter fine‑tuning (QLoRA) on several large protein language models, finding that many model‑task pairs retain over 90% of full fine‑tuning performance while achieving up to 90% GPU memory savings. QLoRA preserves early‑layer representations and induces task‑specific changes in later layers, closely resembling full fine‑tuning with smaller representational shifts. For generative models, 4‑bit quantization largely maintains structural and sequence‑level properties, though token‑level analysis reveals model‑dependent changes in autoregressive output distributions.