← Back to all news
arXiv AI By Muhammad Aurangzeb Ahmad, Kim Shyu, Leon Oliver, Fergus Sleight, Paul Landau

Evaluating AI Generated Summaries for Cancer Patients

Read the original on arXiv AI →

The study evaluates AI-generated summaries for cancer patients using a dual assessment framework that includes human experts and LLM-as-a-judge. Human domain experts—oncology clinicians and patient-facing care staff—assess summary quality on accuracy, clinical relevance, and readability. The research identifies limitations such as omissions and minor inaccuracies, which are then used to iteratively refine prompts, grounding, and safety guardrails.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Computation and Language
Sep 3

Improving Health Literacy through Lay Summarization of Radiological Reports: An Evaluation of BioNER and Retrieval-Augmented Generation

The paper examines how Retrieval-Augmented Generation (RAG) and Named Entity Recognition (NER) affect the quality of lay summaries of radiology reports. Using a framework that extracts clinically relevant findings via NER and grounds them with RAG, the authors evaluate few‑shot and fine‑tuned versions of Qwen and BioBART. Results show that NER consistently improves readability and overall quality, RAG alone offers no benefit and can introduce hallucinations, and the best performance comes from fine‑tuned BioBART with NER.

By Egecan \c{C}elik Evgin, \.Ilknur Karadeniz, Olcay Taner Y{\i}ld{\i}z
llmsragnlpfine-tuningsafety
More like this →
arXiv AI
Sep 15

A primer on evaluation methods for large language models in healthcare

arXiv:2609.14819v1 Announce Type: cross Abstract: Large language models (LLMs) have a growing range of applications in medicine, and their evaluation is critical for ensuring they provide benefit and...

By Suzannah E McKinney, Phuc Vu, Samuel A Justice, Christopher Humphries, Alyssa Pradhan, Timothy J Keyes, Bernardo C Bizzo, Keith J Dreyer, Sarah F Mercaldo, James M Hillis
llmsagentsbenchmarks
More like this →
arXiv AI
Jun 6

Ten Headache Specialists versus Artificial Intelligence for Clinical Literature Summarization: A Critical Evaluation and Comparison

arXiv:2606. 05436v1 Announce Type: new Abstract: Summarizing the latest medical literature to guide clinical decision-making is essential for evidence-based medicine and high-quality patient care.

By Alejandro Lozano, Keiko Ihara, Ping-Hao Yang, Carrie E. Robertson, Jennifer Stern, Allan Purdy, Hsiangkuo Yuan, Pengfei Zhang, Yulia Orlova, Olga Fermo, Jennifer Hranilovich, Fred Cohen, Todd J. Schwedt, Jenelle A. Jindal, Serena Yeung-Levy, Chia-Chun Chiang
llmsragagentsnlpbenchmarks
More like this →
arXiv Computation and Language
3d ago

Abstracts or full text? Evaluating input scope and computational efficiency in biomedical lay summarization

The study investigates whether using only abstracts of biomedical articles can produce lay summaries that match the quality of those derived from full texts, and whether few‑shot learning can replace fine‑tuning large language models. Experiments on two public datasets with two recent models show that abstract‑based summarization outperforms full‑text approaches in readability, factual consistency, semantic similarity, and runtime efficiency. Abstract fine‑tuning is 4.70 times faster and abstract zero‑ and few‑shot prompting is 5.31 times faster than full‑text methods, indicating that abstract‑based prompting reduces computational cost while preserving summary quality.

By Egecan \c{C}elik Evgin, \.{I}lknur Karadeniz, Olcay Taner Y{\i}ld{\i}z
llmsnlpfine-tuning
More like this →