arXiv Machine Learning

Task- and dataset-specific information in protein language models

arXiv:2608. 12090v1 Announce Type: new Abstract: Protein language models (PLMs) have transferred the latest advances from natural language processing to computational biology.

Hugging Face Trending Papers

Aug 12

Task- and dataset-specific information in protein language models

Protein language models (PLMs) have transferred the latest advances from natural language processing to computational biology. These models, trained on large corpora of protein sequence data, are widely used to translate amino acid sequences into latent-space embeddings, ready for use in diverse downstream tasks (DTs).

llms rag nlp

arXiv AI

Jun 12

HD-Prot: A Protein Language Model for Joint Sequence-Structure Modeling with Continuous Structure Tokens

arXiv:2512. 15133v3 Announce Type: replace-cross Abstract: Proteins inherently possess a consistent sequence-structure duality.

By Yi Zhou, Haohao Qu, Yunqing Liu, Shanru Lin, Le Song, Wenqi Fan

llms diffusion fine-tuning efficiency multimodal benchmarks

arXiv AI

Aug 7

Beyond Sequence Order: Syntax-Informed Positional Embeddings for Transformers

arXiv:2608. 06111v1 Announce Type: cross Abstract: Positional embeddings (PE) in Transformers encode token distance and order but are largely agnostic to \textit{syntactic structure}.

By Haris Riaz, Hyungji Kim, Mihai Surdeanu

llms rag benchmarks

arXiv AI

Jul 10

Agentic Neural Architecture Search

arXiv:2607. 07984v1 Announce Type: new Abstract: Neural architecture search (NAS) methods have grown increasingly efficient, yet they remain bounded by manually engineered search spaces that require substantial domain expertise and must be rebuilt for every new task.

By Seokhoon Jeong, Mijung Kim, Taehwan Kim

llms agents computer-vision benchmarks

arXiv AI

Jul 28

LC-SEPLM: long-range contact-supervised adaptation for sequence-only protein representation learning

arXiv:2607. 22777v1 Announce Type: cross Abstract: Protein language models learn transferable sequence representations.

By Chen Wang, Boming Kang, Qinghua Cui

llms fine-tuning benchmarks

arXiv AI

Aug 12

JEPA-DNA: Grounding Genomic Foundation Models through Joint-Embedding Predictive Architectures

arXiv:2602. 17162v3 Announce Type: replace Abstract: Genomic Foundation Models (GFMs) typically rely on Masked Language Modeling (MLM) or Next-Token Prediction (NTP) to learn the "Laws of Nature".

By Ariel Larey, Elay Dahan, Amit Bleiweiss, Raizy Kellerman, Guy Leib, Omri Nayshool, Dan Ofer, Tal Zinger, Dan Dominissini, Gideon Rechavi, Nicole Bussola, Simon Lee, Shane O'Connell, Dung Hoang, Marissa Wirth, Alexander W. Charney, Nati Daniel, Yoli Shavit

rag diffusion benchmarks safety

arXiv AI

Jul 24

Interpretable Embeddings with Sparse Autoencoders: A Data Analysis Toolkit

arXiv:2512. 10092v2 Announce Type: replace Abstract: Analyzing large-scale text corpora is a core challenge in machine learning, crucial for tasks like identifying undesirable model behaviors or biases in training data.

By Nick Jiang, Xiaoqing Sun, Lisa Dunlap, Lewis Smith, Neel Nanda

llms rag

arXiv Machine Learning

Jul 13

Mixture of Probes: Learning from Privileged Modalities in Multimodal LLMs Through Probing

arXiv:2607. 08839v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) are typically designed under the assumption that all modalities available during training will also be accessible at inference.

By Dominick Reilly, Qiyu Wu, Hiromi Wakaki, Srijan Das, Yuki Mistufuji

llms multimodal safety

arXiv Machine Learning

Jun 10

Entropy, Disagreement, and the Limits of Foundation Models in Genomics

arXiv:2604. 04287v2 Announce Type: replace Abstract: Foundation models in genomics have shown mixed success compared to their counterparts in natural language processing.

By Maxime Rochkoulets, Lovro Vr\v{c}ek, Mile \v{S}iki\'c

rag nlp

arXiv AI

Aug 11

LF${}^{2}$AR: Accounting for Layerwise Dynamics to Improve Multimodal Adaptation of Language Models

arXiv:2503. 06211v3 Announce Type: replace-cross Abstract: Text-pretrained language models (LMs) encode rich world knowledge, but adapting them to process and generate perceptual modalities such as audio and images while effectively leveraging that knowledge remains challenging.

By Santiago Cuervo, Adel Moumen, Yanis Labrak, Sameer Khurana, Antoine Laurent, Mickael Rouvier, Phil Woodland, Ricard Marxer

llms multimodal benchmarks safety

arXiv Machine Learning

Jun 26

The Geometry of Updates: Fisher Alignment at Vocabulary Scale

arXiv:2606. 27242v1 Announce Type: new Abstract: Training-free source selection for LLM families with shared vocabularies arises in scientific string domains such as SMILES, protein, and genomic sequences, where candidate corpora share a tokenizer but differ in prediction targets.

By John Sweeney

llms rag nlp safety

arXiv Machine Learning

Aug 4

LangFIR: Discovering Sparse Language-Specific Features from Monolingual Data for Language Steering

arXiv:2604. 03532v2 Announce Type: replace-cross Abstract: Large language models (LLMs) show strong multilingual capabilities, yet reliably controlling the language of their outputs remains difficult.

By Sing Hieng Wong, Hassan Sajjad, A. B. Siddique

llms benchmarks

Related stories