The paper introduces BASSE, a multilingual meta‑evaluation dataset containing 2,040 human‑rated abstractive summaries produced manually or by five LLMs with four prompts. Annotators scored each summary on coherence, consistency, fluency, relevance, and 5W1H using a 5‑point Likert scale. Benchmarking shows proprietary LLM‑judge models best align with human judgments, followed by criteria‑specific automatic metrics, while open‑source judge LLMs perform poorly.
By Jeremy Barnes, Naiara Perez, Alba Bonet-Jover, Bego\~na Altuna
arXiv:2607. 21010v1 Announce Type: new Abstract: Zero-shot summarization using Large Language Models (LLMs) has significantly advanced the abstractive summarization task by producing coherent and fluent summaries.
By Vasudha Bhatnagar, Purnima Bindal, Vikas Kumar, Raj Kumari Bahl
arXiv:2606. 08000v1 Announce Type: cross Abstract: The progress of large language models (LLMs) has fueled claims that model-generated summaries rival or even surpass human-written references, raising questions about whether summarization remains an open research problem.
By Dongqi Liu, Chenxi Whitehouse, Zheng Zhao, Zhuchen Cao, Jian Li, Yabiao Wang
arXiv:2608. 04307v1 Announce Type: cross Abstract: Text summarization is deceptively difficult.
By Karen Lee, Dhanashree Balaram, Seojun Shon, Umair Rasheed
The paper introduces CARPAS, a new task that dynamically refines user-provided aspects for aspect-based summarization in large language models (LLMs). It presents three new datasets and evaluates four prompting strategies, finding that LLMs tend to over-generate aspects, leading to overly long and misaligned summaries. To address this, the authors propose a two-stage framework that first generates lightweight scope guidance before aspect refinement and summarization, which improves focus, reduces over-generation, and enhances performance across all datasets.
By Yong-En Tian, Yu-Chien Tang, An-Zi Yen, Wen-Chih Peng
arXiv:2609.00241v1 Announce Type: new
Abstract: Long documents often distribute important information across extensive narrative passages and multiple tables, making faithful summarization particular...
By Meng Zhou, Wenhao You, Wei Yuan
arXiv:2606. 27226v1 Announce Type: new Abstract: Evaluating LLM outputs remains a major bottleneck in NLP: human evaluation is expensive and slow, lexical metrics correlate poorly with human judgments on open-ended generation, and holistic LLM judges often produce opaque scores that are hard to debug.
By Sangwoo Cho, Kushal Chawla, Pengshan Cai, Zefang Liu, Chenyang Zhu, Shi-Xiong Zhang, Sambit Sahu
arXiv:2607. 15829v1 Announce Type: cross Abstract: Automated essay scoring (AES) enables scalable assessment and timely feedback but remains challenged by transformer input-length limitations, which can cause information loss when processing long essays.
By Haowei Hua
arXiv:2607. 18983v1 Announce Type: cross Abstract: We present AutoJourn, a demonstration system for multi-perspective news generation and bias-aware evaluation using large language models (LLMs).
By Himel Ghosh, Ahmed Mosharafa, Georg Groh
arXiv:2608. 03655v1 Announce Type: cross Abstract: Abstractive summarization models remain vulnerable to factual inconsistency, redundancy, and weak length control.
By Zeyu Wang, Guanghua Wang, Meng Xu
arXiv:2606. 23989v1 Announce Type: cross Abstract: End-to-end large language models (LLMs) produce fluent multi-document summaries but remain prone to hallucination, and the attributions they offer are typically coarse (whole documents or passages) and generated post hoc, leaving each summary statement hard to verify.
By Shuo Guan
arXiv:2606. 05494v3 Announce Type: replace-cross Abstract: Automatic text summarization has become increasingly important due to the rapid growth of digital textual information.
By Ahmed Alansary, Ali Hamdi