arXiv AI
AlphaIdeaBench: Benchmarking Automated Research Idea Generation under Feasibility Constraints with Peer Review Alignment
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Oct 21
IDEAgent: Agentic Quality-Diversity Search for Research Idea Generation
arXiv:2607.22375v2 Announce Type: replace Abstract: Large Language Models (LLMs) have significantly advanced the automation of scientific discovery. However, existing systems share a core limitation:...
arXiv AI
Aug 3
Can AI Evaluate AI Scientists? A Benchmarking Study of Autonomous Research Generation Systems Using Automated Multi-Model Review
arXiv:2607. 28631v1 Announce Type: new Abstract: AI Scientist systems capable of autonomous research have the potential to significantly accelerate scientific discovery.
arXiv AI
Oct 1
AIM: Agentic Idea Management for Automated Research
arXiv:2609.38445v1 Announce Type: new Abstract: Frontier LLMs are increasingly used to automate scientific research through iterative search. We distinguish idea-driven search from solution-driven se...
arXiv AI
Jul 7
ResearchStudio-Idea: An Evidence-Grounded Research-Ideation Skill Suite from ML Conference Outcomes
arXiv:2607. 04439v1 Announce Type: new Abstract: Large language models have made research ideation increasingly accessible, yet effective idea development requires more than generating candidate directions.
arXiv AI
Sep 1
Ideation Arena: Evaluating LLM Generated Research Ideas with Battle-style Human Expert Assessment
Ideation Arena is a battle-style platform that evaluates research ideas generated by large language models (LLMs) and research agents through pairwise human assessment. The system builds shared literature contexts, collects over 6,000 double-blind comparisons from 105 computer science researchers, and constructs an Elo rating leaderboard to rank proposal-stage expert preferences. It also introduces Ideation Arena Eval, a benchmark to test whether automated evaluators align with human preferences, finding that current LLM judges achieve at best 72.56% Soft Accuracy on overall quality.
arXiv AI
Sep 10
AgentIdeaBench: Benchmarking Scientific Ideation in the Agent Era
arXiv:2609.07611v1 Announce Type: new Abstract: Scientific ideation is the capacity to formulate novel and testable hypotheses from scientific evidence, and autonomous AI scientists depend on it. Exi...