arXiv Machine Learning October 23, 2026 By Khouloud Saadi, Di Wang

Same molecule, Different Knowledge: Cross-Identifier Binding Failures in LLMs For chemistry

Read the original on arXiv Machine Learning →

The paper introduces a benchmark that demonstrates that large language models (LLMs) fail to consistently retrieve the same chemical facts when presented with different co‑referring identifiers such as trivial names, IUPAC names, or SMILES. Across models ranging from 7B to 72B parameters, including chemistry‑specialized ones, the authors find that knowledge is not uniformly bound to each identifier, a failure they call cross‑identifier binding failure. They further analyze internal representations and propose a mitigation called Binding Self‑Distillation that transfers knowledge from name‑based to structure‑based routes, enabling direct factual retrieval from SMILES without retraining the model.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Computation and Language

Oct 15

Beyond Benchmark Accuracy: A Diagnostic Test for LLM Chemical Prediction

arXiv:2610.19906v1 Announce Type: new Abstract: LLMs have achieved strong performance on chemical prediction benchmarks. However, existing evaluation methods typically provide target entity identity...

By Yanyi Su, Yu Zeng, Hongshuai Wang, Chenyan Wang, Zhifeng Gao, Xiaohui Yang

llms benchmarks

arXiv AI

Jun 3

Rethinking Molecular Text Representations for LLMs: An Empirical Study

arXiv:2606. 03057v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used for molecular tasks, but it remains unclear which molecular representation to use.

By Arun Raja, Garrett M. Morris, Kian Ming A. Chai

llms nlp benchmarks

arXiv Machine Learning

Jun 5

MolE-RAG: Molecular Structure-Enhanced Retrieval-Augmented Generation for Chemistry

arXiv:2606. 05693v1 Announce Type: new Abstract: Large language models (LLMs) have shown promise for molecular property prediction, but their ability to reason over chemical structures remains limited, as molecular representations such as SMILES differ substantially from the natural language on which LLMs are primarily trained.

By Joey Chan, Wonbin Kweon, Ashley Shin, Niharika Bhattacharjee, Pengcheng Jiang, Yue Guo, Jiawei Han

llms rag fine-tuning

arXiv AI

Aug 12

Multi-Granular Rationale-Guided Molecular LLM for Property Prediction

arXiv:2608. 10480v1 Announce Type: new Abstract: Large language models (LLMs) are widely applied across chemical tasks, such as molecular property prediction, which underpins drug discovery.

By Junwoo Park, Minyoung Shin, Cheol Soon Lee, Sujee Lee

llms fine-tuning

arXiv Machine Learning

Aug 4

SIGMA: Semantic Identifier Grouping for Molecular Autoregression

arXiv:2603. 25062v2 Announce Type: replace Abstract: Autoregressive molecular models assign probability to molecular serializations even though chemical identity is invariant to serialization.

By Xinyu Wang, Fei Dou, Jinbo Bi, Minghu Song

benchmarks safety

Hugging Face Trending Papers

Aug 11

Multi-Granular Rationale-Guided Molecular LLM for Property Prediction

Large language models (LLMs) are widely applied across chemical tasks, such as molecular property prediction, which underpins drug discovery. Molecular LLMs represent a molecule through several modalities, notably a 1D SMILES sequence or a 2D molecular graph.

llms fine-tuning

One email a morning, machine-written

Related stories

Beyond Benchmark Accuracy: A Diagnostic Test for LLM Chemical Prediction

Rethinking Molecular Text Representations for LLMs: An Empirical Study

MolE-RAG: Molecular Structure-Enhanced Retrieval-Augmented Generation for Chemistry

Multi-Granular Rationale-Guided Molecular LLM for Property Prediction

SIGMA: Semantic Identifier Grouping for Molecular Autoregression

Multi-Granular Rationale-Guided Molecular LLM for Property Prediction