arXiv AI By Barbara Kitchenham, Sebasti\'an Pizard, Lech Madeyski, Ronnie de Souza Santos, Martin Shepperd, David Budgen

GUEST: Preliminary Guidelines for Using and Evaluating GenAI Tools to Support Systematic Literature Reviews

Read the original on arXiv AI →

The article presents GUEST, a framework of 51 recommendations for using and evaluating generative AI (GenAI) tools in systematic literature reviews (SLRs). It identifies four methodological categories—SLR practice adjustments, secondary‑study standards, GenAI evaluation practices, and core empirical rigour—to guide researchers in planning, executing, and reporting GenAI‑assisted reviews. The authors conclude that while GenAI cannot replace human oversight for unsupervised SLRs, it can efficiently support repetitive screening and secondary validation tasks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language
Sep 14

PRISMA-LLM: An Empirical Reporting Framework for AI-Assisted Systematic Reviews

The paper introduces PRISMA-LLM, a reporting framework for AI-assisted systematic reviews. It is based on an analysis of 888 review-automation papers, showing a shift toward LLM- and software-driven workflows and inconsistent reporting of evaluation and limitations. The framework separates implementation details from consequence-sensitive evaluation and limitation reporting.

By Miguel Zabaleta, Baihan Lin
arXiv AI
Oct 9

Has LLM Screening Performance Stalled in Software Engineering Systematic Reviews?

The study investigates whether large language models (LLMs) have improved in screening systematic reviews (SRs) within software engineering. Using the SESR‑Eval benchmark and a new smaller dataset, the authors evaluated eight recent LLMs and found only marginal performance gains over older models (average MCC rose from 0.347 to 0.365). The research also examined prompt variations, inter‑model agreement, and the impact of refining inclusion/exclusion criteria, concluding that LLMs still fall short of replacing human reviewers and that future work should explore agent‑based methods, prompt engineering, and further criteria refinement.

By Aleksi Huotala, Miikka Kuutila, Mika M\"antyl\"a
arXiv AI
Jun 30

meta-pipe: An LLM-agent pipeline for end-to-end automated systematic review and meta-analysis

arXiv:2606. 28363v1 Announce Type: cross Abstract: Objective: To describe the architecture and design rationale of meta-pipe, an open-source large language model (LLM)-agent pipeline that integrates the complete systematic review and meta-analysis (SR/MA) workflow -- from literature search through statistical analysis, manuscript generation, and quality assurance -- with mandatory human oversight at critical decision points.

By Hsieh-Ting Lin, Jiunn-Tyng Yeh