arXiv Machine Learning

STAR-VAE: A Scalable Latent-Variable Transformer for Controllable Molecular Generation

STAR-VAE is a Transformer-based variational autoencoder that uses SELFIES encoding and a bidirectional encoder with an autoregressive decoder pretrained on 79 million PubChem molecules. It incorporates a property signal to jointly condition the prior, posterior, and decoder, and employs LoRA adapters for fine‑tuning on small datasets without altering the backbone. The model achieves 100 % validity and near‑perfect novelty in MOSES sampling, low KL divergence on several GuacaMol descriptors, strong synthetic‑accessibility conditioning, and effective docking‑score control across multiple protein targets, while also enabling scaffold recovery and diverse label‑conditioned generation on ChEMBL targets.

arXiv Machine Learning

1d ago

StabilityArc: Decoding Protein Sequence Embeddings into Generalizable Stability Landscapes

StabilityArc is a method that decodes protein sequence embeddings into generalizable stability landscapes. It uses a shared RoPE transformer to map frozen ESMC-600M residue representations into an Lx20 matrix of substitution effects, with a symmetric, contact-aware residual to predict epistasis. In extensive leave-one-protein-out tests on 134,794 ProteinGym variants, StabilityArc achieves a Spearman correlation of 0.7134, surpassing the best zero‑shot baseline, and further improves Kermut’s performance when used as a prior.

By Aaron L. Feller, Andrew D. Ellington, Claus O. Wilke

llms rag

arXiv Machine Learning

Jul 28

MS-GPT: Rethinking MS/MS De Novo Structure Elucidation as Spectrum-Induced Posterior Querying of a Molecule-Language Model

arXiv:2607. 23607v1 Announce Type: new Abstract: Molecular structure elucidation from tandem mass spectra (MS/MS) is a central inverse problem in analytical chemistry.

By Xin Zhao, Yumin Liu, Zhuo Li, Weichu Zheng, Feng Zhu, Xiaokang Yang, Yaohui Jin, Yanyan Xu

llms fine-tuning safety

arXiv Machine Learning

Jun 26

Training-Free Generation of Protein Sequences from Small Family Alignments via Stochastic Attention

arXiv:2603. 14717v2 Announce Type: replace Abstract: Generating novel protein sequences that respect a family's statistical constraints typically requires training deep generative models on thousands to millions of examples.

By Jeffrey D. Varner

llms safety

arXiv AI

Jun 10

Flexible Flows for Biological Sequence Design

arXiv:2606. 10543v1 Announce Type: cross Abstract: Designing functional biological sequences requires navigating vast discrete spaces under strict evolutionary and biophysical constraints.

By Yogesh Verma, Dani Korpela, Harri L\"ahdesm\"aki, Vikas Garg

diffusion benchmarks

arXiv Machine Learning

Sep 4

SimpleDesign: A Joint Model for Protein Sequence and Structure Codesign

SimpleDesign is a single-stage, end-to-end model for joint protein sequence and structure design that eliminates the need for multi-stage training. It combines discrete cross-entropy for sequences with a regression objective for structures, using a Mixture-of-Transformer architecture to handle modality-specific processing while maintaining global self-attention. Trained on over 2 million sequence-structure pairs, SimpleDesign achieves strong performance on co-design and unconditional generation benchmarks.

By Jiarui Lu, Yuyang Wang, Yizhe Zhang, Jiatao Gu, Navdeep Jaitly, Joshua M. Susskind, Miguel \'Angel Bautista

llms diffusion multimodal benchmarks

arXiv Machine Learning

Jun 9

In-Context Learning for Latent Space Bayesian Optimization

arXiv:2606. 09664v1 Announce Type: new Abstract: Bayesian optimization (BO) is a central tool for sample-efficient design, and latent-space Bayesian optimization (LSBO) extends it to structured objects such as molecules and proteins.

By Tuan A. Vu, Harri L\"ahdesm\"aki, Julien Martinelli

llms diffusion benchmarks

arXiv Machine Learning

Aug 21

PETA:Parameter-Efficient Test-Time Adaptation for Virtual Screening

arXiv:2608. 19906v1 Announce Type: new Abstract: Accurately ranking active ligands for a target protein pocket from massive chemical libraries remains a central challenge in virtual screening.

By Jia-Qi Lin, Yinghua Yao, Chang-Dong Wang, Yew-Soon Ong, Yuangang Pan

rag diffusion benchmarks

arXiv Machine Learning

1d ago

Analysis of Quantized and Efficiently Adapted Protein Language Models

The study evaluates 4‑bit quantization and low‑rank adapter fine‑tuning (QLoRA) on several large protein language models, finding that many model‑task pairs retain over 90% of full fine‑tuning performance while achieving up to 90% GPU memory savings. QLoRA preserves early‑layer representations and induces task‑specific changes in later layers, closely resembling full fine‑tuning with smaller representational shifts. For generative models, 4‑bit quantization largely maintains structural and sequence‑level properties, though token‑level analysis reveals model‑dependent changes in autoregressive output distributions.

By Ilan Yaniv Zeisler, Sebastian Clancy, Pouriya Bayat, Saaim Raad, Ivan Kraskov, Matthew Xie, Vivian White, Spencer Perkins, Serena Singh, Sepehr Bayat, Keith Pardee

llms fine-tuning efficiency

arXiv Machine Learning

Jul 21

GeneSpeak-FP: Target and Compound Retrieval from Observed Cell-Level Perturbation Signatures

arXiv:2607. 17671v1 Announce Type: new Abstract: Large-scale single-cell perturbation atlases make it possible to ask an inverse question: given an observed transcriptional response, which annotated targets and compounds in a fixed library are most consistent with that response?

By Kseniia Vaniushkina, Jeongmin Lim, Jinyong Park

llms rag safety

arXiv Machine Learning

Jun 4

BBOmix: A Tabular Benchmark for Hyperparameter Optimization of Unsupervised Biological Representation Learning

arXiv:2606. 05139v1 Announce Type: new Abstract: The rapid advancement of high-throughput sequencing has led to large, high-dimensional omics datasets.

By Luca Thale-Bombien, Jan Ewald, Ralf K\"onig, Aaron Klein

fine-tuning efficiency benchmarks

arXiv Machine Learning

1d ago

A Large Scale Investigation of Scaling Limits in Chemical Language Models

The paper reports a large-scale, compute-controlled study of Chemical Language Models (CLMs) involving over 30,000 experiments across different molecular representations, tokenizations, model sizes, datasets, and architectures. It finds clear scaling trends in pretraining loss but shows that these improvements do not translate into proportional gains in goal-directed molecular design, with chemical syntax saturating early while semantic properties develop more slowly. The authors release a new suite of models, NovoMolGen, that achieves state-of-the-art results in drug discovery tasks, highlighting a disconnect between representation learning and downstream design and calling for new pretraining paradigms that target chemical semantics.

By Roshan Balaji, Kamran Chitsaz, Quentin Fournier, Nirav Pravinbhai Bhatt, Sarath Chandar

llms benchmarks

arXiv Machine Learning

Jul 1

Can Tabular In-Context Learners Generalize to Biomolecular Property Prediction?

arXiv:2606. 31126v1 Announce Type: new Abstract: Predicting biomolecular properties from limited labeled data is a central bottleneck in protein engineering and small-molecule design.

By Davy Guan, Lu Zhang, Asiri Wijesinghe, Allen Zhu, He Zhao, Helen Power, F. Hafna Ahmed, Andrew Warden, Cheng Soon Ong, Daniel M. Steinberg

llms safety

Related stories