arXiv AI
Jun 11
On the Limits of LLM-as-Judge for Scientific Novelty Assessment
arXiv:2606. 12071v1 Announce Type: cross Abstract: LLMs are increasingly used to generate and judge scientific ideas.
An Axiomatic Benchmark for Evaluation of Scientific Novelty Metrics
arXiv:2604. 15145v2 Announce Type: replace Abstract: The rigorous evaluation of the novelty of a scientific paper is, even for human scientists, a challenging task.
Related stories
arXiv Machine Learning
Aug 4
LLM generation novelty through the lens of semantic similarity
arXiv:2510. 27313v3 Announce Type: replace Abstract: Generation novelty is a key indicator of an LLM's ability to generalize, yet measuring it against full pretraining corpora is computationally challenging.
arXiv AI
1d ago
Beyond Correctness: Toward Automated Novelty Verification with Lean 4
arXiv:2608. 14669v1 Announce Type: new Abstract: Artificial intelligence systems applied to mathematics verify correctness but not novelty: an automatically generated theorem can compile in Lean without errors and yet be an already known result.
arXiv AI
Jul 31
Do Methods Support the Claims? Intra-Paper Verification for Peer Review
arXiv:2607. 26066v1 Announce Type: cross Abstract: The growing volume of scientific submissions has motivated interest in using large language models (LLMs) to assist peer review.
arXiv AI
Jul 2
Heuresis: Search Strategies for Autonomous AI Research Agents Across Quality, Diversity and Novelty
arXiv:2606. 25198v2 Announce Type: replace Abstract: Autonomous AI Research promises to accelerate the scientific progress of machine learning.
arXiv AI
6d ago
Text Corpora as Concept Fields: Black-Box Hallucination and Novelty Measurement
arXiv:2605. 05103v3 Announce Type: replace-cross Abstract: We introduce the \textbf{Concept Field} of a text corpus: a local drift field with pointwise uncertainty, estimated in sentence-embedding space from the deltas between consecutive sentences.
arXiv AI
Jun 9
Contemporary AI lacks the imagination to diverge or negate in science
arXiv:2606. 08251v1 Announce Type: cross Abstract: Bold projections that artificial intelligence will accelerate scientific discovery have raced ahead of evidence from working scientists, and the field still lacks large-scale, scientist-in-the-loop tests of these claims.
arXiv AI
Jun 19
Benchmarking Agentic Review Systems
arXiv:2606. 19749v1 Announce Type: new Abstract: A new class of agentic review systems are emerging as a remedy to the pressure placed on peer review systems by AI-assisted research, but it is unclear how they should be evaluated.
arXiv AI
Aug 3
Can AI Evaluate AI Scientists? A Benchmarking Study of Autonomous Research Generation Systems Using Automated Multi-Model Review
arXiv:2607. 28631v1 Announce Type: new Abstract: AI Scientist systems capable of autonomous research have the potential to significantly accelerate scientific discovery.
Hugging Face Trending Papers
Jun 10
Automated Creativity Evaluation of Language Models Across Open-Ended Tasks
Large language models (LLMs) have achieved remarkable progress in language understanding, reasoning, and generation, sparking growing interest in their creative potential. Realizing this potential requires systematic and scalable methods for evaluating creativity across diverse tasks.
arXiv AI
Jul 1
RARE: Redundancy-Aware Retrieval Evaluation Framework for High-Similarity Corpora
arXiv:2604. 19047v2 Announce Type: replace-cross Abstract: Existing QA benchmarks typically assume distinct documents with minimal overlap, yet real-world retrieval-augmented generation (RAG) systems operate on corpora such as financial reports, legal codes, and patents, where information is highly redundant and documents exhibit strong inter-document similarity.
arXiv AI
Jun 29
Towards Automating Scientific Review with Google's Paper Assistant Tool
arXiv:2606. 28277v1 Announce Type: cross Abstract: Artificial intelligence is driving a revolution in scientific discovery, accelerating everything from hypothesis generation to mathematical theorem proving.