arXiv Machine Learning
RAGenome: Scaling Retrieval-Based Genomic Language Models to Long Contexts
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jul 13
TheBioCollection: Unified Pre-Training Scale LLM Corpus for Biology
arXiv:2607. 08803v1 Announce Type: cross Abstract: The push toward large language models for biology (BioLM) has created a need for training corpora that can endow models with a genuine understanding of biology.
arXiv AI
Aug 12
JEPA-DNA: Grounding Genomic Foundation Models through Joint-Embedding Predictive Architectures
arXiv:2602. 17162v3 Announce Type: replace Abstract: Genomic Foundation Models (GFMs) typically rely on Masked Language Modeling (MLM) or Next-Token Prediction (NTP) to learn the "Laws of Nature".
arXiv Machine Learning
Sep 22
Tool-Augmented On-Policy Distillation for LLM Domain Adaptation in Sequence-Based Omics Tasks
The paper introduces OmicsBench, a new reasoning benchmark for multi‑omics sequences that includes 1,160 expert‑validated questions across DNA regulation, RNA processing, and protein function tasks, requiring traceable evidence chains. Evaluation of 17 large language models shows that scientific LLMs, while more accurate in classification, often lack valid evidence, suggesting shortcut learning. To address this, the authors propose tool‑augmented on‑policy distillation (TA‑OPD), a post‑training method that improves both evidence grounding and predictive performance across five Qwen3.5 models of varying sizes.
arXiv Machine Learning
Sep 30
LEMON-ZEST: Evolution-Informed Tokenization for Efficient Protein Language Modeling
arXiv:2609.37675v1 Announce Type: new Abstract: Protein Language Models (PLMs) have made remarkable progress following scaling laws established in natural language processing across sequence- and str...
arXiv Machine Learning
Sep 11
When do cheap embeddings beat protein language models? A theoretically-grounded hashing sketch for biological sequence classification
The paper introduces Murmur2Vec, a lightweight, alignment‑free embedding that uses k‑mer counts hashed with MurmurHash to create a compact representation for biological sequences. It provides a full theoretical analysis, including bias/variance formulas, a Johnson–Lindenstrauss‑style concentration bound, and an excess‑risk bound that clarifies the trade‑off between hash‑table size and classifier performance. Empirically, Murmur2Vec matches or surpasses a fine‑tuned 650M‑parameter ESM‑2 protein language model across several classification tasks, including SARS‑CoV‑2 spike lineage and HIV‑1 Env subtype identification.
arXiv Machine Learning
Aug 19
Quantifying Memorization and Privacy Risks in Genomic Language Models
The paper introduces a comprehensive privacy evaluation framework for genomic language models (GLMs) that quantifies memorization risks using perplexity-based detection, canary sequence extraction, and membership inference. By planting canary sequences at different repetition rates in synthetic and real datasets, the authors systematically assess how repetition, model capacity, and training dynamics affect memorization across various GLM architectures. The study demonstrates that GLMs do memorize training data to varying degrees and that no single attack method fully captures this risk, highlighting the necessity of multi-vector privacy auditing for genomic AI systems.