arXiv Machine Learning

A Survey of LLM Prompt Datasets: Taxonomy, Linguistic Patterns, and Practical Uses

The paper presents a survey of 129 public large language model (LLM) prompt datasets, totaling over 1.22 TB and 673 million instances, and introduces a unified taxonomy for them. By analyzing seven datasets in depth, the authors identify lexical, syntactic, and semantic patterns that differentiate prompts from general text, and evaluate these patterns for tasks such as prompt filtering, source domain routing, and response quality assessment. They demonstrate that a 63‑dimensional linguistic feature set extracted on a CPU can match over 91 % of the F1 score of GPU‑based sentence embeddings while halving latency, and that structural features can effectively route prompts across datasets, though they may negatively impact response quality when prompt length is controlled.

arXiv AI

Aug 24

SCOPE: A Generative Approach for LLM Prompt Compression

SCOPE is a training‑free generative prompt‑compression framework that reduces LLM input length by chunking a prompt into semantically coherent segments, rewriting each chunk to be more concise, and then reconstructing a coherent prompt. Unlike token‑removal methods, SCOPE’s chunk‑level rewriting preserves critical information and text coherence, and includes optimization techniques for finer‑grained control of compression ratios. Extensive evaluations on question‑answering and summarization tasks show that SCOPE consistently outperforms selective compression baselines, especially at high compression ratios.

By Tinghui Zhang, Yifan Wang, Daisy Zhe Wang

llms rag nlp

arXiv Computation and Language

4d ago

Learning a Single Token to Replace Long System Prompts in LLMs

arXiv:2511.23271v2 Announce Type: replace Abstract: Long system prompts are widely used to steer Large Language Models (LLMs), but repeatedly processing them at inference time is inefficient and cons...

By Jiancheng Dong, Pengyue Jia, Jingyu Peng, Maolin Wang, Yuhao Wang, Lixin Su, Xin Sun, Shuaiqiang Wang, Dawei Yin, Xiangyu Zhao

llms

Hugging Face Trending Papers

Aug 13

Prompts in the Wild: A Large Analyzed Collection of Transactional Prompts in Code

The behavior of contemporary generative Large Language Models (LLMs) is directly shaped by prompts, unstructured texts that describe the desired output and model behavior. In this paper we argue that prompts are linguistic objects that merit investigation in their own right.

llms

arXiv AI

Aug 24

PromptResponse: Optimizing Prompts for LLM Coding Tasks

arXiv:2608.21074v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used in research workflows and software development pipelines, yet their output remains sensitive to in...

By Erik Thureck, Robert K\"uhnen, Tim Jacobowitz

llms benchmarks safety

arXiv AI

Jul 31

DialectLLM: A Dialect-Aware Dialog[ue] Generation Framework Beyond Standard American English

arXiv:2601. 22888v4 Announce Type: replace-cross Abstract: More than 80% of the 1.

By Jio Oh, Paul Vicinanza, Thomas Butler, Steven Euijong Whang, Dezhi Hong, Amani Namboori

llms benchmarks data-engineering

arXiv AI

Jun 9

Knowing How to Edit: Reliable Evaluation Signals for Diagnosing and Optimizing Prompts at Query Level

arXiv:2511. 19829v3 Announce Type: replace Abstract: Prompt optimization has become a central mechanism for eliciting strong performance from LLMs, and recent work has made substantial progress by proposing diverse prompt evaluation metrics and optimization strategies.

By Ke Chen, Yifeng Wang, Hassan Almosapeeh, Haohan Wang

llms benchmarks

arXiv AI

Jul 29

Every Time I Hire a Linguist, Inference Costs Go Down: On Linguistic Rules as Effective Prompt Compressors

arXiv:2607. 25335v1 Announce Type: cross Abstract: Prompt compression shortens LLM input to reduce inference cost, yet existing methods score token importance through LM forward passes.

By Jianfei Ma, Zhaoxin Feng, Emmanuele Chersoni, Si Chen

llms efficiency

arXiv Machine Learning

Jul 10

Prompt Compression via Activation Aggregation

arXiv:2607. 08399v1 Announce Type: cross Abstract: Large language models process prompts by propagating activations through dozens of layers before generating a response.

By Thibaud Ardoin, Semira Einsele, Evis Bregu, Gerhard Wunder

llms

arXiv AI

Aug 12

A Cost-Efficient Routing Pipeline for Multilingual Short-Text Classification Using Small Language Models

arXiv:2608. 10939v1 Announce Type: cross Abstract: Multilingual short-text classification supports operational systems such as content moderation, customer support routing, and intent recognition, yet aggregate evaluation often hides large differences between high-resource and low-resource languages.

By Wajdi Ben Saad, Safa Madiouni

llms nlp fine-tuning benchmarks

arXiv AI

Jul 24

PromptPack: Scaling LLM Annotation Agents for Online Recommendation

arXiv:2607. 20528v1 Announce Type: new Abstract: Online recommendation platforms increasingly use Large Language Models (LLMs) to extract structured features from ad creatives.

By Sebastian Koralewski, Merwan Barlier, Yulia Stolin, Bla\v{z} \v{S}krlj

llms agents efficiency benchmarks

arXiv AI

Jun 2

A Unified Evaluation-Instructed Framework for Query-Dependent Prompt Optimization

arXiv:2511. 19829v2 Announce Type: replace Abstract: Most prompt-optimization methods refine a single static template, making them ineffective in complex and dynamic user scenarios.

By Ke Chen, Yifeng Wang, Hassan Almosapeeh, Haohan Wang

arXiv AI

Aug 24

Beyond Prompt Engineering: A Systematic Analysis of Prompt Lexical Sensitivity and Its Impacts on Quality

The paper investigates how small lexical changes in prompts can cause large performance swings in large language models. Using a dataset of 132,000 prompt variants, the authors uncover a scaling law linking higher average task performance to lower variance and greater robustness. They identify domain-specific terminology and explicit action directives as key linguistic factors that stabilize prompts, and propose an automated Prompt-Refining Agent that reduces performance variance by 40.7% in code generation while maintaining or improving mean performance.

By Qipeng Xie, Zi Liang, Jiafei Wu, Yufei Chen, Weizheng Wang, Wenao Ma, Zhong Ming, Haiqin Yang, Kaishun Wu

llms agents

Related stories