arXiv AI
Aug 7
An Axiomatic Benchmark for Evaluation of Scientific Novelty Metrics
arXiv:2604. 15145v2 Announce Type: replace Abstract: The rigorous evaluation of the novelty of a scientific paper is, even for human scientists, a challenging task.
Measuring the Novelty of Biomedical Papers Using the Latent Distances between Knowledge Units
The paper proposes a new method for measuring the novelty of biomedical research by calculating latent distances between knowledge units—specifically MeSH terms—using three types of relationships: network, semantic, and hierarchical. It demonstrates that each relationship captures distinct distances and that combining all three yields a more accurate novelty assessment than existing metrics. Validation on a large PLoS ONE dataset and a H1 Connect dataset shows stronger alignment with peer judgments compared to prior indicators.
Related stories
arXiv AI
Jul 15
Research Novelty in Information Systems Journals After ChatGPT: Differences Across Institutional Language Contexts
arXiv:2603. 22510v2 Announce Type: replace-cross Abstract: Large language models are increasingly used in scholarly work, yet it remains unclear whether their productivity gains are accompanied by changes in research novelty.
arXiv AI
Jul 31
Do Methods Support the Claims? Intra-Paper Verification for Peer Review
arXiv:2607. 26066v1 Announce Type: cross Abstract: The growing volume of scientific submissions has motivated interest in using large language models (LLMs) to assist peer review.
arXiv Computation and Language
Sep 1
MemoNoveltyAgent: A Historical Research Memory-Aware Agent Workflow for Paper Novelty Assessment
arXiv:2603.20884v4 Announce Type: replace Abstract: To alleviate the heavy burden of paper screening, researchers increasingly rely on existing AI agents, such as AI reviewers or DeepResearch, for pa...
arXiv AI
Sep 12
NovGauge: A Fine-Grained Benchmark for Diagnosing LLMs' Capability in Paper Novelty Assessment
NovGauge is a new benchmark designed to diagnose large language models’ ability to assess scientific paper novelty. It contains 619 paper pairs and 50 multi-paper sets, each labeled along three dimensions—task, problem, and method—by experts from ICLR reviewer overlap claims and survey co-citations. The study evaluates 18 LLMs, revealing high hallucination rates and weak evidence grounding, with the best model achieving only 43‑72% verified F1 across dimensions.
arXiv AI
Jun 11
On the Limits of LLM-as-Judge for Scientific Novelty Assessment
arXiv:2606. 12071v1 Announce Type: cross Abstract: LLMs are increasingly used to generate and judge scientific ideas.
arXiv Computation and Language
Sep 23
A Semantic Approach to the Academic Publishing Network: Document Vector Representations and Hybrid Structural-Semantic Fusion over OpenAlex Data
arXiv:2609.26218v1 Announce Type: cross Abstract: Structural graph analysis of the academic publishing network captures the topological relationships between entities but does not see the content of...
arXiv Computation and Language
Aug 25
How Does Research Evolve? Tracing Cross-Domain Trajectories in NLP, ML, and CV Through Claim-Grounded Typed Citations
arXiv:2606.22342v2 Announce Type: replace Abstract: How does research evolve, and can we trace it at the level of individual claims? Scientific progress is not simply a uniform accumulation of facts....
arXiv AI
Sep 7
Hakken: Predicting future discoveries to fill the gaps in today's knowledge
Hakken is a domain‑agnostic system that predicts and explains future scientific discoveries by combining transformer‑based models trained on temporal knowledge graphs with large language model semantic knowledge. It identifies novel relationships between scientific concepts that extend beyond the deductive hull of existing knowledge and provides explanations to help scientists assess these predictions. In the biomedical domain, Hakken set a new benchmark for time‑aware multi‑label relation prediction, generated 1.5 million high‑confidence hypotheses about aging, and experimentally confirmed two predictions that revealed previously undocumented interactions relevant to drug discovery.
arXiv Computation and Language
Aug 27
Think-Probe-Respond: Improving Large Language Models as Judges of Research Idea Novelty
The paper introduces Think‑Probe‑Respond (TPR), a lightweight method to improve large language models’ ability to judge the novelty of research ideas. It identifies a systematic bias where models tend to label ideas as "medium novel" despite generating human‑like rationales, and shows that probing hidden states during reasoning and conditioning the final response on these probes boosts novelty judgment accuracy by 22.30%. TPR effectively reduces the medium‑novelty bias across strong baseline models.
arXiv Computation and Language
Sep 15
Measuring the Creativity of Frontier LLMs in Automated Research
arXiv:2609.14057v1 Announce Type: new Abstract: Frontier LLMs are increasingly capable of conducting automated research, yet their creativity in this setting has not been systematically evaluated. In...
Hugging Face Trending Papers
Jun 21
How Does Research Evolve? Tracing Cross-Domain Trajectories in NLP, ML, and CV with Claim-Grounded Typed Citations
How does research evolve, and what substrate would let us forecast where it goes next? Scientific progress is not simply a uniform accumulation of facts: ideas extend prior methods, address known limitations, realize proposed future directions, and sometimes dispute earlier claims.