arXiv:2606. 11166v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly described as performing at the level of human experts on knowledge economy tasks.
By George Perrett, Javae Elliott, Jennifer Hill, Marc Scott
arXiv:2609.24516v1 Announce Type: new
Abstract: In recent years, large language models (LLMs) have emerged as a popular alternative for evaluation. Often referred to as LLMs as judges (LLJs), these s...
By Khaoula Chehbouni, Melina Medjdoub, Florian Carichon, Golnoosh Farnadi, Jackie Chi Kit Cheung
The study investigates whether large language models (LLMs) have improved in screening systematic reviews (SRs) within software engineering. Using the SESR‑Eval benchmark and a new smaller dataset, the authors evaluated eight recent LLMs and found only marginal performance gains over older models (average MCC rose from 0.347 to 0.365). The research also examined prompt variations, inter‑model agreement, and the impact of refining inclusion/exclusion criteria, concluding that LLMs still fall short of replacing human reviewers and that future work should explore agent‑based methods, prompt engineering, and further criteria refinement.
By Aleksi Huotala, Miikka Kuutila, Mika M\"antyl\"a
arXiv:2609.23201v1 Announce Type: new
Abstract: Benchmark scores increasingly influence the development, marketing, and selection of large language models (LLMs). Yet an overall score is interpretabl...
By Danial Amin
arXiv:2610.19185v1 Announce Type: new
Abstract: Evaluation of large language models (LLMs) remains largely pre-scientific: it involves many design decisions, with little guidance on how to make them....
By Shenbin Qian, David Samuel, Jind\v{r}ich Helcl, Andrey Kutuzov, Stephan Oepen
arXiv:2609.33153v2 Announce Type: replace-cross
Abstract: Reported improvements from tools and reusable skills in large language model agents refer to different comparisons. This critical narrative r...
By Shuyang Zhang