arXiv Machine Learning

Retrieval-guided Twin Fusion with Similarity-aware Contrast for Molecule-Text Alignment

arXiv:2608. 16005v1 Announce Type: new Abstract: This paper studies the problem of molecule-text alignment, which aims to project molecules and their textual descriptions into a joint latent space for downstream tasks including molecule search and molecular property prediction.

Hugging Face Trending Papers
Jun 29

PromptGNN-sim: Deep Fusion and Alignment of GNN and LLMs for Text-Attributed Graph Learning

Text-Attributed Graphs (TAGs) combine textual semantics with graph structure and are central to many graph learning tasks. However, existing fusion methods often treat text and structure as separate inputs in a shallow, one-way pipeline, which limits deep interaction between modalities and weakens performance under sparse connectivity or cross-graph generalisation.

arXiv AI
Aug 26

MolEmb: Multimodal Large Language Models Can Be Strong Molecular Embedding Models

MolEmb is a lightweight framework that adapts multimodal large language models (MLLMs) to serve as general molecular embedding models. By aligning molecular profiles with textual descriptions in a shared embedding space using a bidirectional contrastive objective, MolEmb produces embeddings conditioned on both a molecular profile and a natural‑language semantic context. The model performs competitively on molecular property prediction and enables cross‑modal molecule‑text retrieval, while the newly introduced MolCAR benchmark demonstrates that context‑aware molecular embedding is largely a data property of the supervision.

By Xinjian Zhao, Xiangru Jian, Yaoyao Xu, Xiaozhuang Song, Wei Pang, Lei Bai, Tianshu Yu
arXiv Computation and Language
Aug 25

Aligning Biomedical Texts and Knowledge Graphs: A Systematic Comparison of Lightweight Alignment Strategies

The paper introduces a unified framework for aligning biomedical text with knowledge graphs using a lightweight projection learned via contrastive learning, keeping the text encoder and KG embedding model frozen. It evaluates six design choices—text encoder, KG embedding, projection head, triple composition, training direction, and hard‑negative sampling—on a newly created CTD‑Align corpus of 22K chemical‑gene interaction pairs linked to PubMed passages. The study finds that triple composition and training direction have the largest impact, while simpler linear projections over concatenated subject, predicate, and object embeddings yield the best performance.

By Artem Bisliouk, Elizaveta Nosova, Heiko Paulheim, Andreea Iana, Rita T. Sousa
arXiv Machine Learning
5d ago

MSAlign: Aligning Molecule and Mass Spectra representations for Metabolite Identification

The paper introduces MSAlign, a lightweight model that aligns frozen foundation models for mass spectra (DreaMS) and molecules (MolDeBERTa) to improve metabolite identification from MS/MS spectra. It presents a unified framework for representation alignment and contrastive learning, demonstrates that a score fusion strategy further boosts performance at minimal cost, and addresses evaluation challenges by quantifying distribution shift in data splitting strategies. All resources, including datasets, splits, and code, are publicly released to promote reproducible research.

By Paul Krzakala, Gabriel Melo, Camille Lan\c{c}on, Charlotte Laclau, R\'emi Flamary, Etienne Th\'evenot, Florence d'Alch\'e-Buc
arXiv AI
Aug 28

MedFG-VQA: Low-Frequency Memory and Graph Attention for Lightweight Medical VQA

MedFG-VQA is a lightweight medical visual question answering framework that uses a memory bank to enhance low‑frequency DCT features and graph‑enhanced cross‑attention for visual‑textual alignment. It introduces Frequency‑Memory Fusion to retrieve and fuse low‑frequency information from a learnable memory bank, and Graph‑Aware Cross‑Attention to refine cross‑modal features via graph convolution. The authors also create SynMed‑VQA, a synthetic dataset of over 2 million QA pairs across nine imaging modalities, and show that MedFG‑VQA matches or outperforms larger models on several biomedical VQA benchmarks while keeping computational costs low.

By Haowen Gu, Gensheng Pei, Zeren Sun, Mingwu Ren, Xiangbo Shu, Yazhou Yao, Fumin Shen
arXiv AI
Sep 3

Subcellularly Resolved Single-Cell Embedding Learning with Transcriptomic data, Protein Structure and Localization Information

The paper introduces a multimodal framework that learns subcellularly resolved cell embeddings by integrating RNA expression profiles, protein sequence representations, and protein structural information using a cross‑attention architecture. This approach models interactions within distinct subcellular compartments, producing fine‑grained embeddings that capture both molecular expression patterns and functional protein properties. It is presented as the first method to jointly incorporate transcriptomic data, sequence, and structural knowledge for subcellularly resolved cell representation.

By Zhen Zhou, Jiachen Li, Yuan Liu, Xiaoyong Pan, Hong-Bin Shen