arXiv AI

From Citations to Contributions: LLM-Assisted Credit Scoring of Research Articles

The paper proposes a new method for credit scoring research articles that distinguishes between a paper’s original contribution and the prior work it builds upon. It introduces a hierarchical ‘contribution tree’ framework that conserves importance across a document’s structure and separates original from citation-derived credit. Large language models are employed as noisy comparative estimators to scale the analysis, and the approach is extended to collections of articles via weighted citation graphs to produce corpus-level contributions and normalized influence scores.

arXiv Computation and Language
Sep 2

The Scientific Contribution Graph: Automated Literature-based Technological Roadmapping at Scale

The paper introduces the Scientific Contribution Graph, a large-scale resource that extracts 6 million scientific contributions from 655 k open-access papers across multiple disciplines and links them with 36 million prerequisite edges. It frames automated technological roadmapping as the task of identifying contributions and their prerequisites, and presents a new scientific prerequisite prediction task where models forecast which existing technologies enable future discoveries. The authors report that current models achieve a 0.48 MAP score on temporally-filtered backtesting, indicating rapid progress in this area.

By Peter A. Jansen
arXiv AI
Sep 10

Attribution in Scientific Literature: New Benchmark and Methods

The paper introduces REASONS, a benchmark of 12,723 sentence-level citation instances across 12 arXiv subject categories, to evaluate scientific citation attribution under different evidence conditions. It proposes a dual-metric framework—Abstention Rate (AR) and Hallucination Rate (HR)—to balance reliability and responsiveness. Experiments with proprietary and open-source LLMs across various prompting and retrieval settings show that advanced Retrieval-Augmented Generation (RAG) reduces hallucinations but increases abstention, while adversarial metadata can push hallucination rates above 85%. Human evaluation confirms a high ratio of factual hallucinations to acceptable paraphrases, underscoring the need for systems that can appropriately abstain under uncertainty.

By Deepa Tilwani, Yash Saxena, Seyedali Mohammadi, Ankur Padia, Edward Raff, Amit Sheth, Srinivasan Parthasarathy, Manas Gaur
arXiv Computation and Language
Sep 10

Towards Automatic Evolution Tree Generation from Citation Graphs

The paper introduces EvoTree, a staged framework for automatically generating evolution trees from citation graphs. It separates backbone learning from temporal refinement, using a graph-aware encoder and hierarchical clustering to build a stable taxonomy, then fine-tunes temporally to attach marginal papers under monotonic-path constraints, and finally labels concepts with an LLM without changing the topology. The authors release an annotated benchmark across 11 AI subfields and report that EvoTree outperforms baselines in NMI, citation-direction accuracy, concept purity, and marginal-paper detection.

By Zexing Zhao, Yuntong Hu, Liang Zhao