arXiv AI June 11, 2026 By Daniel Commey

When Generic Prompt Improvements Hurt: Evaluation-Driven Iteration for LLM Applications

arXiv:2601. 22025v2 Announce Type: replace-cross Abstract: Evaluating Large Language Model (LLM) applications differs from conventional software testing because outputs are probabilistic, semantically variable, and sensitive to prompt and model changes.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language

Sep 14

PRISMA-LLM: An Empirical Reporting Framework for AI-Assisted Systematic Reviews

The paper introduces PRISMA-LLM, a reporting framework for AI-assisted systematic reviews. It is based on an analysis of 888 review-automation papers, showing a shift toward LLM- and software-driven workflows and inconsistent reporting of evaluation and limitations. The framework separates implementation details from consequence-sensitive evaluation and limitation reporting.

By Miguel Zabaleta, Baihan Lin

llms

arXiv AI

Jun 3

Acceptance-Test-Driven Evaluation Protocols for Business-Centric LLM Systems

arXiv:2606. 02755v1 Announce Type: cross Abstract: Large language model (LLM) applications are increasingly expected to satisfy deterministic institutional requirements while relying on probabilistic generative components.

By Eric Liang

llms agents fine-tuning benchmarks safety

arXiv AI

Jun 10

LLM-Based Code Documentation Generation and Multi-Judge Evaluation

arXiv:2606. 09852v1 Announce Type: cross Abstract: High-quality source code documentation is vital yet often neglected, especially in critical domains like healthcare where reliability and maintainability are essential.

By Ikbel Ghrab, Mohamed Dhieb, Ismail Khenissi, Ines Abdeljaoued-Tej

llms agents safety

arXiv AI

Jun 16

WorkflowPerturb: Calibrated Stress Tests for Evaluating Multi-Agent Workflow Metrics

arXiv:2602. 17990v2 Announce Type: replace Abstract: Multi-agent LLM systems that generate structured workflows from natural-language requests are now deployed in production across cloud automation, DevOps, and enterprise process orchestration.

By Madhav Kanda, Sharad Agarwal, Rodrigo Fonseca, Alok Gautam Kumbhare, Pedro Las-Casas

llms agents benchmarks

arXiv AI

Aug 11

$A^2E$ : An End-to-End Agent Auditing Engine

arXiv:2608. 07346v2 Announce Type: replace Abstract: With the rapid advancement of large language models (LLMs), harnesses have become essential infrastructure for deploying agents across a wide range of domains.

By Haoning Wang, Mingxun Zhang, Chenyue Yu, Yingjun Shang, Xia Hu, Guanchu Wang, Na Zou

llms agents

arXiv AI

Aug 10

An End-to-End Agent Auditing Engine

arXiv:2608. 07346v1 Announce Type: new Abstract: With the rapid advancement of large language models (LLMs), harnesses have become essential infrastructure for deploying agents across a wide range of domains.

By Haoning Wang, Mingxun Zhang, Chenyue Yu, Yingjun Shang, Xia Hu, Guanchu Wang, Na Zou

llms agents

One email a morning, machine-written

Related stories

PRISMA-LLM: An Empirical Reporting Framework for AI-Assisted Systematic Reviews

Acceptance-Test-Driven Evaluation Protocols for Business-Centric LLM Systems

LLM-Based Code Documentation Generation and Multi-Judge Evaluation

WorkflowPerturb: Calibrated Stress Tests for Evaluating Multi-Agent Workflow Metrics

$A^2E$ : An End-to-End Agent Auditing Engine

An End-to-End Agent Auditing Engine