arXiv Machine Learning By Micha{\l} Kulczykowski, Rafa{\l} {\L}ab\k{e}dzki

Self-Supervised Pretraining of Molecular Graph Encoders with LeJEPA

Read the original on arXiv Machine Learning →

The study investigates whether self‑supervised pretraining improves molecular graph neural networks by adapting the LeJEPA architecture to molecular graphs. While pretraining enhances learned representations and a frozen probe outperforms random initialization on tasks such as ogbg‑molhiv, it does not consistently boost finetuning performance across different data splits. Combining pretrained embeddings with 1024‑bit Morgan fingerprints yields modest gains, indicating that pretraining provides complementary information best exploited at the feature level.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI
Sep 17

Procedural Pretraining for Molecular Property Prediction

The paper proposes a three‑stage training pipeline that begins with procedural pretraining on abstract, procedurally generated data, followed by molecular pretraining on SMILES, and finally downstream fine‑tuning for molecular property prediction. Experiments show that procedural pretraining improves downstream performance—e.g., a 4.8% error reduction on Lipophilicity—especially when labeled data are scarce, and that the benefit peaks at an intermediate procedural training budget. Analysis indicates that transferable knowledge resides mainly in attention layers, while feed‑forward layers may over‑specialize.

By Moritz Friedemann, Zachary Shinnick, Philip Torr, Bruno Andreis
arXiv Machine Learning
Sep 22

CurvFlow-DTA: dual-graph discrete Ricci curvature flow for drug--target affinity prediction

CurvFlow-DTA introduces a dual-graph discrete Ricci curvature flow framework for drug–target affinity prediction, replacing static curvature with weighted Forman curvature flow on both drug and protein residue–residue contact graphs. The method precomputes a label‑independent flow trajectory for each entity and uses a pair‑conditioned selector to guide a dual‑branch Flow‑GINE, leveraging frozen ESM‑2 residue representations. Experiments on Davis and KIBA datasets show significant improvements over the Ricci‑GraphDTA baseline, with reductions in mean squared error of up to 19.9% in warm‑start and 27.4% in cold‑start settings, and higher concordance indices across benchmarks.

By Jicheng Ma, Yunyan Yang, Juan Zhao, Liang Zhao
arXiv Machine Learning
Aug 6

Geometry-Informed Parameter-Efficient Fine-Tuning of Pre-trained Molecular GNNs for Blood-Brain Barrier Permeability Prediction

arXiv:2608. 04257v1 Announce Type: new Abstract: Blood-brain barrier permeability (BBBP) prediction is a critical screening task in central nervous system drug discovery, where candidate molecules must be assessed for whether they can cross, or should be prevented from crossing, the blood-brain barrier.

By Marco Vieto Vega, Long D. Nguyen, Binh P. Nguyen
arXiv Machine Learning
Sep 7

WEECFP-SuRGE: Wide Embedded Extended Connectivity Fingerprint with Substructure Rotary Graph-distance Encoding

WEECFP-SuRGE introduces a 1024‑dimensional, parameter‑free continuous fingerprint that distributes each Morgan substructure across about thirty‑two signed positions in a single vector. The accompanying transformer architecture applies Substructure Rotary Graph‑distance Encoding (SuRGE), a RoPE‑like rotation based on molecular shortest‑path graph distance, to the fingerprint tokens. In benchmark tests, a seven‑model blend of this architecture achieves top rankings on the TDC ADMET leaderboard and outperforms classical fingerprints on most MoleculeNet regression tasks, while its tokenization scheme is shown to be near‑lossless and highly efficient for positional memory.

By Robert Epps
arXiv Machine Learning
Jun 9

Do Larger Models Really Win in Drug Discovery? A Benchmark Assessment of Model Scaling in AI-Driven Molecular Property and Activity Prediction

arXiv:2604. 26498v3 Announce Type: replace Abstract: The rapid growth of molecular foundation models and large language models (LLMs) has encouraged a scale centred view of AI in drug discovery, in which larger pretrained models are expected to supersede compact cheminformatics models.

By Jinjiang Guo, Sheng Ding