Tracing the Influence of Retrieved Literature on Scientific Ideation in LLMs

Large language models (LLMs) are increasingly used to generate research ideas from scientific literature, but the impact of the retrieval strategy on the ideas produced is unclear. This study isolates that contribution by comparing five retrieval strategies and a no‑literature baseline across 45,000 ideas in 15 machine learning topics and ten open‑weight models. The results show that while all retrieval strategies reduce diversity within a single set of papers, they increase diversity when ideas from different sets are pooled; effects on quality and novelty are modest but topic‑dependent, and a JSD‑based attribution method reveals that most ideas are driven by one or two key papers, with more varied source attribution correlating with greater idea diversity.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

• llms