arXiv Computation and Language
Interpretable Text Embeddings Based on Hierarchical Questions
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv Computation and Language
Aug 24
Granuscore: A Reference-Free Measure of Granularity for Text Analysis and Question Answering
Granuscore is a reference‑free metric that measures the granularity of text by exploiting the structure of a hierarchical embedding space. It successfully reproduces known hierarchical orderings on the Granola‑EQ dataset, distinguishes granularity across different discourse contexts, and explains sentence‑specificity variations beyond sentence length. The authors also apply Granuscore to four question‑answering benchmarks, revealing systematic differences in granularity among questions, gold answers, and model outputs, thereby offering a new lens for assessing QA dataset difficulty.
arXiv AI
Jun 16
JE-IRT: A Geometric Lens on LLM Abilities through Joint Embedding Item Response Theory
arXiv:2509. 22888v2 Announce Type: replace Abstract: Standard LLM evaluation practices compress diverse abilities into single scores, obscuring their inherently multidimensional nature.
arXiv AI
Jul 8
From Voting to Agent Collaboration: Answer-Type-Aware LLM Pipelines for BioASQ 14b
arXiv:2607. 06452v1 Announce Type: cross Abstract: Biomedical question answering requires not only accurate extraction of information from scientific literature but also reliable integration of evidence across multiple documents.
arXiv AI
Jun 10
RankLLM: Weighted Ranking of LLMs by Quantifying Question Difficulty
arXiv:2602. 12424v2 Announce Type: replace-cross Abstract: Benchmarks establish a standardized evaluation framework to systematically assess the performance of large language models (LLMs), facilitating objective comparisons and driving advancements in the field.
arXiv Computer Vision
Aug 24
Generative Embedding Benchmark: How Much Information Survives in a Dense Embedding?
The paper introduces the Generative Embedding Benchmark (GEB), which evaluates how much content from an embedding can be recovered by a decoder that only has access to the frozen embedding and a question, without the original image or intermediate features. GEB uses a curated visual‑question‑answering dataset with 1,800 development and 900 test items covering natural images, scene text, and visual documents. Experiments on seven public embedding models show that visual‑only scores range from 28.25 to 33.21, while joint image‑question encoding boosts scores up to 65.56, revealing that generative readout uncovers information bottlenecks not captured by traditional separability‑based benchmarks.
arXiv Computation and Language
Oct 27
LatentRAG: Latent Reasoning and Retrieval for Efficient Agentic RAG
arXiv:2605.06285v2 Announce Type: replace Abstract: Single-step retrieval-augmented generation (RAG) provides an efficient way to incorporate external information for simple question answering tasks...