arXiv Computation and Language September 7, 2026 By Neeladri Bhuiya, Shib Sankar Dasgupta, Andrew McCallum, Haw-Shiuan Chang

PROMPT2BOX:Improving LLM Weakness Discovery and Specificity Estimation by Uncovering Entailment Structure among Prompts

Read the original on arXiv Computation and Language →

The paper introduces Prompt2Box, a method that embeds prompts into a box embedding space to capture both semantic similarity and specificity relations, addressing the limitation of traditional vector embeddings that conflate topical similarity with specificity. Using a trained encoder on existing and synthesized datasets, Prompt2Box achieves significant improvements, reducing specificity prediction error by 45% over a prompt-length baseline and identifying 13.5% more LLM weaknesses in hierarchical clustering compared to vector baselines. The authors also present a novel dimension‑reduction technique for visualizing and comparing box embeddings, and provide the code on GitHub.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Computation and Language

3d ago

The Garden of Forking Prompts: How Users Explore Narrative Space in Story Generation

arXiv:2609.14677v1 Announce Type: new Abstract: Large language models (LLMs) have changed the way people engage with stories. Drawing on public chatbot logs, we can see that when users generate stori...

By Advait Deshmukh, Nora Benedict, Melanie Walsh, Maria Antoniak

llms benchmarks

arXiv AI

Jul 24

Interpretable Embeddings with Sparse Autoencoders: A Data Analysis Toolkit

arXiv:2512. 10092v2 Announce Type: replace Abstract: Analyzing large-scale text corpora is a core challenge in machine learning, crucial for tasks like identifying undesirable model behaviors or biases in training data.

By Nick Jiang, Xiaoqing Sun, Lisa Dunlap, Lewis Smith, Neel Nanda

llms rag

arXiv Machine Learning

Aug 28

A Survey of LLM Prompt Datasets: Taxonomy, Linguistic Patterns, and Practical Uses

The paper presents a survey of 129 public large language model (LLM) prompt datasets, totaling over 1.22 TB and 673 million instances, and introduces a unified taxonomy for them. By analyzing seven datasets in depth, the authors identify lexical, syntactic, and semantic patterns that differentiate prompts from general text, and evaluate these patterns for tasks such as prompt filtering, source domain routing, and response quality assessment. They demonstrate that a 63‑dimensional linguistic feature set extracted on a CPU can match over 91 % of the F1 score of GPU‑based sentence embeddings while halving latency, and that structural features can effectively route prompts across datasets, though they may negatively impact response quality when prompt length is controlled.

By Yuanming Zhang, Yan Lin, Arijit Khan, Huaiyu Wan

llms rag

arXiv AI

Jul 22

Hyperdimensional Probe: Decoding LLM Representations via Vector Symbolic Architectures

arXiv:2509. 25045v3 Announce Type: replace-cross Abstract: Despite their capabilities, Large Language Models (LLMs) remain opaque with limited understanding of their internal representations.

By Marco Bronzini, Carlo Nicolini, Bruno Lepri, Jacopo Staiano, Andrea Passerini

llms rag efficiency safety

arXiv Computation and Language

Aug 28

TopiCLEAR: Adaptive embedding clustering for interpretable topic discovery from short texts

TopiCLEAR is a framework that clusters document or sentence embeddings using adaptive dimensionality reduction to uncover low‑dimensional geometric structures that correspond to human‑interpretable topics. The method is evaluated on four benchmark datasets, showing strong agreement with human annotations, especially for short and informal texts. A Twitter case study demonstrates that TopiCLEAR yields more interpretable topics than LDA, recovering both annotated topic structure and coherent sub‑topics.

By Aoi Fujita, Taichi Yamamoto, Yuri Nakayama, Ryota Kobayashi

llms rag benchmarks safety

arXiv Machine Learning

Aug 13

Task- and dataset-specific information in protein language models

arXiv:2608. 12090v1 Announce Type: new Abstract: Protein language models (PLMs) have transferred the latest advances from natural language processing to computational biology.

By Roman Joeres, Ilya Senatorov, Olga V. Kalinina

llms rag nlp