arXiv Machine Learning
Oct 5

Beyond Random Splits: Evaluating Drug-Target Affinity Models Under Chemically and Biologically Motivated Distribution Shifts Copy

The study investigates how different distribution shifts affect the evaluation of drug‑target affinity (DTA) models, using 718,800 drug‑protein pairs from ChEMBL and BindingDB. It compares a baseline Morgan‑fingerprint + protein‑CNN model with 12 architectures that combine four drug representations and three ESM‑2 interaction modes, finding that mean validation RMSE rises significantly under protein‑cluster and dual out‑of‑distribution (OOD) scenarios. Model rankings remain consistent across chemical shifts but diverge under protein and dual OOD, indicating that architecture selection depends on the specific extrapolation context rather than just average error or training‑set size. Why it matters: The results suggest that DTA benchmarks should be designed to reflect the chemical and target shifts expected in real deployment, ensuring that chosen models perform reliably when applied to new chemical series or protein targets.

By Minjae Chung, Clara Li, Malar Paavai Muthukumaran, Shaunna Wang, Aniket Ramkrishnan Iyer, Shaun Qien Yeau Tan, Harinishree Sathu, Micky C. Nnamdi, J. Ben Tamo, Benoit Louis Marteau, May Dongmei Wang