Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

32,584 stories · RSS feed

arXiv Computation and Language
Sep 15

Chemical Chain-of-Thought Functions as a Hallucination-Prone Molecular Scratchpad

Chemical reasoning language models are expected to produce faithful chain-of-thought (CoT) explanations when answering chemistry tasks, but across four model families and twelve tasks, hallucinations are widespread and largely independent of answer correctness. Attribution analyses reveal that these models use a shared scratchpad function: Chem‑R and ether‑0 rely on fragmented SMILES drafts, while ChemDFM‑R emphasizes scaffold, positional, and naming cues. Perturbing Chem‑R’s SMILES sketches degrades generation, indicating that structural drafts can be causally load‑bearing even when verbal structural claims are largely inert.

By Jiatong Li, Yuxuan Ren, Weida Wang, Xiaoyong Wei, Yatao Bian
arXiv AI
Sep 15

ModaLens: Measuring Image Sensitivity in Report-Conditioned Medical VLMs

arXiv:2609.15635v1 Announce Type: cross Abstract: A radiology report can already answer a clinical question, so it is hard to tell whether a vision-language model also uses the image. ModaLens, a pai...

By Sebasti\'an Andr\'es Cajas Ord\'o\~nez, Maximin Lange, Quang Bui, Anqi Peter Li, Felipe Ocampo Osorio, Rafi Al Attrach, Kushul Reddy Palakala, Sahil Kapadia, Zakaria Laouabdia Sellami, Xinyue Zhang, Ashley Zhang, Leo Anthony Celi
arXiv AI
Sep 15

Vulnerability Localization Benchmark: Measuring Agentic Security Analysis at Repository Scale

arXiv:2609.15939v1 Announce Type: cross Abstract: Language-model agents increasingly operate over complete software repositories, yet cybersecurity evaluations primarily measure whether they can dete...

By Aman Priyanshu, Supriti Vijay, Kimia Majd, Xuhong He, Fraser Burch, Takahiro Matsumoto, Jianliang He, Baturay Saglam, Arthur Goldblatt, Zhuoran Yang, Amin Karbasi