Granuscore is a reference‑free metric that measures the granularity of text by exploiting the structure of a hierarchical embedding space. It successfully reproduces known hierarchical orderings on the Granola‑EQ dataset, distinguishes granularity across different discourse contexts, and explains sentence‑specificity variations beyond sentence length. The authors also apply Granuscore to four question‑answering benchmarks, revealing systematic differences in granularity among questions, gold answers, and model outputs, thereby offering a new lens for assessing QA dataset difficulty.
By Lukas Ellinger, Alexander Fichtl, Miriam Ansch\"utz, Georg Groh
arXiv:2509. 22888v2 Announce Type: replace Abstract: Standard LLM evaluation practices compress diverse abilities into single scores, obscuring their inherently multidimensional nature.
By Louie Hong Yao, Nicholas Jarvis, Tiffany Zhan, Saptarshi Ghosh, Linfeng Liu, Tianyu Jiang
arXiv:2607. 06452v1 Announce Type: cross Abstract: Biomedical question answering requires not only accurate extraction of information from scientific literature but also reliable integration of evidence across multiple documents.
By Taeyun Roh, Eunha Lee, Wonjune Jang, Sohyun Chung, Junha Jung, Jaewoo Kang
arXiv:2602. 12424v2 Announce Type: replace-cross Abstract: Benchmarks establish a standardized evaluation framework to systematically assess the performance of large language models (LLMs), facilitating objective comparisons and driving advancements in the field.
By Ziqian Zhang, Xingjian Hu, Yue Huang, Kai Zhang, Ruoxi Chen, Yixin Liu, Qingsong Wen, Kaidi Xu, Xiangliang Zhang, Neil Zhenqiang Gong, Lichao Sun
The paper introduces the Generative Embedding Benchmark (GEB), which evaluates how much content from an embedding can be recovered by a decoder that only has access to the frozen embedding and a question, without the original image or intermediate features. GEB uses a curated visual‑question‑answering dataset with 1,800 development and 900 test items covering natural images, scene text, and visual documents. Experiments on seven public embedding models show that visual‑only scores range from 28.25 to 33.21, while joint image‑question encoding boosts scores up to 65.56, revealing that generative readout uncovers information bottlenecks not captured by traditional separability‑based benchmarks.
By Yun Li, Biao Yang, Peixi Wu, Yunhao Zhou, Mingzhou Jiang, Wei Yuan, Fan Yang, Wenwu Ou
arXiv:2605.06285v2 Announce Type: replace
Abstract: Single-step retrieval-augmented generation (RAG) provides an efficient way to incorporate external information for simple question answering tasks...
By Yijia Zheng, Marcel Worring
arXiv:2606. 06197v1 Announce Type: cross Abstract: Question answering (QA) systems have achieved notable progress with the advent of large language models (LLMs).
By Hafez Abdelghaffar, Ahmed Alansary, Ali Hamdi
arXiv:2603.01690v3 Announce Type: replace-cross
Abstract: While dense biomedical embeddings achieve strong performance, their opaque dimensions limit transparency in biomedical NLP. Recent question-b...
By Yixuan Tang, Zhenghong Lin, Yandong Sun, Wynne Hsu, Mong Li Lee, Anthony K. H. Tung
The paper introduces KBevo, a co‑evolving framework that simultaneously builds a structured knowledge base and performs reasoning over it for knowledge‑intensive question answering. By optimizing both components end‑to‑end with QA outcome rewards, the system improves the quality and connectivity of the knowledge base, leading to higher answer reachability and better compositional factual reasoning. Compared to standard retrieval baselines, KBevo offers greater controllability and improved factual accuracy.
By Ryan Thomas Noonan, Linxi Zhao, Menghan Xu, Akanksha Sarkar, Mihir Mishra, Dongyoung Go, Kilian Q. Weinberger, Yoav Artzi, Jennifer J. Sun
arXiv:2606. 10460v1 Announce Type: cross Abstract: Recent large language models (LLMs) have shown rapid progress in reading-based question answering (QA), where evidence is explicitly provided or can be trivially retrieved.
By Haonan Wang, Jiaxiang Liu, Yurong Liu, Austin Senna Wijaya, Tianle Zhou, Eden Wu, Yijia Chen, Wanting You, Reya Vir, Daniela Pinto, Grace Fan, Yusen Zhang, Juliana Freire, Eugene Wu
arXiv:2607. 04625v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) streamlines long-document understanding by leveraging retrieval mechanisms to restrict input images to a highly curated subset.
By Junyu Xiong, Yonghui Wang, Rongjian Gu, Chenyu Liu, Bing Yin, Wengang Zhou, Houqiang Li
Tables are ubiquitous across diverse domains, yet reasoning over them remains a significant challenge for modern large language models (LLMs). Current approaches typically linearize tables into sequen...