arXiv:2607. 01233v1 Announce Type: cross Abstract: LLMs are increasingly used to brainstorm research ideas, but existing evaluations mostly judge individual ideas by novelty, feasibility, or expert preference.
arXiv:2605. 10574v3 Announce Type: replace Abstract: As artificial intelligence advances, models are not improving uniformly.
arXiv:2606. 25198v2 Announce Type: replace Abstract: Autonomous AI Research promises to accelerate the scientific progress of machine learning.
arXiv:2606. 08251v1 Announce Type: cross Abstract: Bold projections that artificial intelligence will accelerate scientific discovery have raced ahead of evidence from working scientists, and the field still lacks large-scale, scientist-in-the-loop tests of these claims.
arXiv:2606. 12071v1 Announce Type: cross Abstract: LLMs are increasingly used to generate and judge scientific ideas.
arXiv:2603. 22510v2 Announce Type: replace-cross Abstract: Large language models are increasingly used in scholarly work, yet it remains unclear whether their productivity gains are accompanied by changes in research novelty.
Doc2LoRA introduces a method to represent scientific papers as LoRA adapters generated by a hypernetwork, enabling each point in the resulting vector space—whether a real paper or a mixture—to be interpreted by a large language model (LLM). The approach allows both search and generation: it achieves competitive search performance with models like SPECTER2 and EmbeddingGemma, while also enabling the LLM to label subfields and generate abstracts that smoothly transition between papers. The invertible transform ensures every point maps back to an LLM, making the embeddings usable for both querying and creative idea generation.
The paper introduces SciMuse, an AI system that generates personalized research ideas by combining a knowledge graph of 58 million papers with a large language model. A large-scale evaluation involving over 100 research group leaders across disciplines rated more than 4,400 ideas, yielding modest overall interest scores but showing that 24.9% were rated highly. The study also demonstrates that graph-derived features can predict idea interest and can be used to control idea properties, offering a new methodology for generating and assessing scientific ideas.
ScholarCatalyst is a new benchmark that evaluates how well AI systems can retrieve research papers that inspire new work. The dataset was created by having 184 lead authors of 207 recent computer science papers annotate which earlier papers helped their projects, providing detailed rationales. The benchmark tests retrieval from the literature available at the start of a project, revealing that current agentic search and even advanced models like Claude Fable 5.1 perform only modestly better than simple embedding retrieval.
arXiv:2604. 15145v2 Announce Type: replace Abstract: The rigorous evaluation of the novelty of a scientific paper is, even for human scientists, a challenging task.
arXiv:2610.02022v2 Announce Type: replace Abstract: AI-based ideation systems are often evaluated on the novelty of their ideas, and that judgment is increasingly delegated to large language models....
The paper introduces a method for training large language models to generate diverse, high‑quality scientific ideas grounded in input documents. By using a set‑level reinforcement learning objective that rewards ideas for being distinct from earlier ones, the authors train an 8B model that outperforms Claude Opus 4.8 and GPT‑6 Astra on a contamination‑free benchmark. Human experts rate the model’s idea sets as equally promising while containing slightly more unique ideas, and automated experiments show comparable executability.