arXiv AI

Measuring the State of Open Science in Transportation Using Large Language Models

arXiv:2601. 14429v2 Announce Type: replace-cross Abstract: Open science initiatives have strengthened scientific integrity and accelerated research progress across many fields, but the state of their practice within transportation research remains under-investigated.

arXiv AI
Jul 24

From Static Bibliometrics to Dynamic Knowledge Graphs: An LLM-Powered Framework for Modernizing Science, Technology, and Innovation (STI) Analytics

arXiv:2607. 21327v1 Announce Type: cross Abstract: Bibliometric indicators - citation counts, h-indexes, co-authorship networks - have long anchored science, technology, and innovation (STI) analytics, yet suffer from temporal lag, semantic shallowness, and an inability to capture the non-linear dynamics of contemporary knowledge ecosystems.

By Muhsen Hammoud
arXiv Computation and Language
3d ago

Drift Inspector: Exploring and Measuring Scientific Drift with Atomic Contribution Claims

Drift Inspector is an open‑source system that extracts Atomic Contribution Claims (ACCs) from scientific abstracts using an LLM, then clusters these claims over time to map how a research field evolves. Applied to six years of EMNLP, the tool reveals a shift from classic NLP tasks toward LLM‑era capabilities such as reasoning and multimodality—trends that keyword or whole‑abstract counts miss. The pipeline has also processed the entire ACL Anthology, yielding 346,000 claims from 80,000 abstracts across 423 venues, with human‑validated extraction and clustering aligned to an external taxonomy.

By Vsevolod Karimov, Stepan Ostarkov, Anastasia Poroshina, Anatoly Frolov, Alexander Panchenko
arXiv AI
Sep 16

AquiLLM: Evaluating Faithfulness in Open-Weight RAG-LLM Systems for Scientific Research

arXiv:2609.16519v1 Announce Type: new Abstract: Scientific research increasingly relies on large, heterogeneous data sources, motivating interest in retrieval-augmented generation (RAG) systems that...

By Bernie Boscoe, Srinath Saikrishnan, Vikram Seenivasan, Jack Stark, Andrew Lizarraga, Morgan Himes, Jonathan Soriano, PJ Allen, Tuan Do
arXiv AI
Sep 21

From Code Archival to Knowledge Graph: Bridging Software Heritage, COAR Notify and Wikidata

The paper introduces an end‑to‑end pipeline that harvests, validates, and models links between scholarly articles and their source code from journals such as JOSS, SoftwareX, and IPOL, as well as SIGMOD ARI reproducibility reports. It produces a curated set of 4,397 DOI‑repository pairs and defines two Wikidata‑based application profiles—one for articles and one for software—aligned with schema.org and CodeMeta. Using these profiles, the authors created 4,182 new Wikidata software items linked to their papers, while only 82 repositories were previously represented, and they demonstrate compatibility with the COAR Notify protocol for future live enrichment.

By Camillo Carlo Pellizzari di San Girolamo, Francesco Tosoni
arXiv Computation and Language
Sep 25

CodeGraph: Open-Taxonomy Knowledge Graph for Source Code with Wikidata Grounding

The paper introduces CodeGraph, an open‑taxonomy knowledge graph that semantically annotates source code by extracting entities such as algorithms, paradigms, design patterns, and application domains from millions of files. Using a specialized large language model and a three‑stage Wikidata linking process, the authors ground these entities in Wikidata and construct a graph with about 158 million nodes and 1 billion typed edges across 14 programming languages. A quality‑assurance protocol combining human evaluation and an LLM‑as‑a‑judge filter quantifies annotation precision.

By Federico Pennino, Andrea Gurioli, Stefano Zacchiroli, Maurizio Gabbrielli, Paolo Ferragina
arXiv Computation and Language
Sep 2

The Scientific Contribution Graph: Automated Literature-based Technological Roadmapping at Scale

The paper introduces the Scientific Contribution Graph, a large-scale resource that extracts 6 million scientific contributions from 655 k open-access papers across multiple disciplines and links them with 36 million prerequisite edges. It frames automated technological roadmapping as the task of identifying contributions and their prerequisites, and presents a new scientific prerequisite prediction task where models forecast which existing technologies enable future discoveries. The authors report that current models achieve a 0.48 MAP score on temporally-filtered backtesting, indicating rapid progress in this area.

By Peter A. Jansen