arXiv AI

Hakken: Predicting future discoveries to fill the gaps in today's knowledge

Hakken is a domain‑agnostic system that predicts and explains future scientific discoveries by combining transformer‑based models trained on temporal knowledge graphs with large language model semantic knowledge. It identifies novel relationships between scientific concepts that extend beyond the deductive hull of existing knowledge and provides explanations to help scientists assess these predictions. In the biomedical domain, Hakken set a new benchmark for time‑aware multi‑label relation prediction, generated 1.5 million high‑confidence hypotheses about aging, and experimentally confirmed two predictions that revealed previously undocumented interactions relevant to drug discovery.

arXiv AI
6d ago

Learning Scientific Trajectories to Predict Future Biomedical Discoveries

ForeSCITE is a framework that uses large language models to analyze temporal patterns in biomedical conclusions. It clusters published conclusions, estimates a direction that separates earlier from more recent findings within each cluster, and projects conclusion embeddings along these directions. The projected coordinates are decoded into natural‑language hypotheses, and retrospective evaluations show that ForeSCITE expands semantic coverage, introduces novel biomedical entity pairs, and identifies plausible CAR‑T hypotheses that experts deem biologically relevant.

By Chia-Hsuan Chang, Brian Ondov, Jiang Lv, Lan Jiang, Yunmai Wang, Haerin Song, Xueqing Peng, Qiaozhu Mei, Xiaolei Su, Hua Xu
arXiv Computation and Language
Sep 14

HypoKG: Evidence-Disciplined Biomedical Hypothesis Generation Beyond Endpoint Knowledge

The paper introduces HypoKG, a unified biochemical knowledge graph built from KEGG, Rhea, and UniProt, and uses it to benchmark 13,200 biomedical hypotheses generated by six large language models (LLMs). By varying the biological information provided—source enzyme only, full biological path, or source and disease endpoint—the study finds that LLMs produce higher-scoring hypotheses when given minimal information, but these are less evidence‑grounded. When supplied with the full biological path, the models generate hypotheses that align more closely with known mechanistic relationships, a phenomenon the authors term evidence‑disciplined reasoning, which is confirmed by shuffling intermediate path steps. "whyItMatters":"The study demonstrates that knowledge graphs can both uncover novel disease–enzyme pairs and guide LLMs to reason more accurately from evidence, improving the reliability of AI‑generated biomedical hypotheses."

By Dominic Okonkwo, Adetayo Okunoye, Ismailcem Budak Arpinar
arXiv Machine Learning
4d ago

Do Language Models Know What Compelling Scientific Evidence Looks Like?

The paper introduces EDB-Bio, a benchmark of 96,000 episodes drawn from real biological studies—drug‑perturbation transcriptomics, CRISPR screens, and GWAS—to test whether language models can weigh empirical evidence and predict hypothesis truth. Initial results show that both open‑weight and frontier models are systematically underconfident, matching a trivial base‑rate forecaster in Brier score, but post‑training with proper scoring rules improves calibration and discrimination. In a downstream experiment‑selection task, a calibrated model (Ibis0) outperforms the best frontier model by 2.5×, and Bayes‑factor analysis reveals how models update beliefs, align with expert relevance judgments, and expose biases.

By Conor Igoe, Gabriel Grand, Zane Koch, Albert Bou, Eli Moss, Michaela M. Hinks, Samuel G. Rodriques, Andrew White, Siddharth Narayanan
arXiv Machine Learning
Sep 22

Tool-Augmented On-Policy Distillation for LLM Domain Adaptation in Sequence-Based Omics Tasks

The paper introduces OmicsBench, a new reasoning benchmark for multi‑omics sequences that includes 1,160 expert‑validated questions across DNA regulation, RNA processing, and protein function tasks, requiring traceable evidence chains. Evaluation of 17 large language models shows that scientific LLMs, while more accurate in classification, often lack valid evidence, suggesting shortcut learning. To address this, the authors propose tool‑augmented on‑policy distillation (TA‑OPD), a post‑training method that improves both evidence grounding and predictive performance across five Qwen3.5 models of varying sizes.

By Jie Ying, Zhefan Wang, Zihong Chen, Zhengqing Li, Jinzhe Li, Gang Li, Jian Liu, Fang Hu, Tao Luo, Zhonghang Yuan, Wanli Ouyang, Stan Z. Li, Fan Yang, Nanqing Dong