arXiv AI
SCIRIGOR:Evaluating Open-Ended Scientific Analysis Beyond Final Scores
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 30
SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agents
arXiv:2603. 29139v2 Announce Type: replace Abstract: Recent advances in large language models (LLMs) have enabled agentic systems to translate natural-language intent into executable scientific visualization (SciVis) tasks.
arXiv AI
2d ago
DISCERN: Can AI Agents Work Like Scientists and Guide Discovery?
arXiv:2609.33357v2 Announce Type: replace Abstract: Reliable automated research requires agents to vet data, verify analyses, and generate hypotheses grounded in trustworthy evidence, potentially red...
arXiv AI
1d ago
Tracing the Cascade: A Topology-Aware Evaluation Framework for Scientific Agent Hallucinations
arXiv:2608.00711v2 Announce Type: replace Abstract: Large language model (LLM) agents are increasingly deployed in scientific research, where reliability is critical and the underlying knowledge is d...
arXiv AI
Sep 7
TruthInsightBench: An Evidence-Grounded Benchmark for Automated Evaluation of Open-Ended Scientific Discovery Agents
TruthInsightBench is a new benchmark designed to evaluate automated scientific discovery agents by presenting them with 40 blind tasks drawn from peer‑reviewed studies across ten domains. Each task provides only a neutral objective and frozen data, withholding source conclusions, expected values, and analysis paths, forcing agents to determine which claim the data support. A fixed LLM‑based judge scores agents on evidentiary maturity across six dimensions, using 29 artifact‑grounded items, enabling fully automated, repeatable evaluation without human grading.
arXiv AI
Jul 17
Building Agent Harnesses for Scientific Curation from Multimodal Sources
arXiv:2606. 21005v2 Announce Type: replace Abstract: Scientific discovery workflows often depend on structured curation from the literature.
arXiv AI
Jun 11
Can AI Agents Synthesize Scientific Conclusions?
arXiv:2606. 11337v1 Announce Type: new Abstract: Scientific AI agents increasingly retrieve evidence, reason across sources, and synthesize conclusions used in consequential decisions.