The paper investigates how small lexical changes in prompts can cause large performance swings in large language models. Using a dataset of 132,000 prompt variants, the authors uncover a scaling law linking higher average task performance to lower variance and greater robustness. They identify domain-specific terminology and explicit action directives as key linguistic factors that stabilize prompts, and propose an automated Prompt-Refining Agent that reduces performance variance by 40.7% in code generation while maintaining or improving mean performance.
By Qipeng Xie, Zi Liang, Jiafei Wu, Yufei Chen, Weizheng Wang, Wenao Ma, Zhong Ming, Haiqin Yang, Kaishun Wu
arXiv:2606. 24841v1 Announce Type: new Abstract: Prompt-based learning has emerged as a dominant paradigm in natural language processing.
By Ahmad Pouramini, Hesham Faili
GFlowPO introduces a probabilistic framework for optimizing language model prompts by treating prompt search as posterior inference over latent prompts, regularized by a meta-prompted reference-LM prior. The method first fine‑tunes a lightweight prompt‑LM using an off‑policy Generative Flow Network objective that reuses past prompt evaluations for sample‑efficient exploration. It then applies a training‑free Dynamic Memory Update to refine the meta‑prompt with diverse and top‑performing prompts, leading to consistent improvements over recent discrete prompt optimization baselines on few‑shot classification, instruction induction, and question‑answering tasks.
By Junmo Cho, Suhan Kim, Sangjune An, Minsu Kim, Dong Bok Lee, Heejun Lee, Sung Ju Hwang, Hae Beom Lee
The paper proposes a two‑stage framework for expanding and refining a set of prompts for language models under a limited evaluation budget. In the first stage, it generates a finite candidate set of prompts represented by moderate‑dimensional vectors. The second stage builds a surrogate model of prompt scores and sequentially selects prompts for evaluation, proving consistency of this procedure and introducing a refinement step that searches beyond the initial candidates. Numerical experiments show the method outperforms baseline prompt selection, especially when evaluation calls are constrained.
By Haoting Zhang, Jinghai He, Rhonda Righter, Zeyu Zheng
arXiv:2609.39927v1 Announce Type: new
Abstract: Prompt optimization improves the performance of language-model systems on downstream tasks by refining their prompts. Classical methods evaluate prompt...
By Junyang Chen, Zecheng Wang, Jingbang Chen
arXiv:2507. 09839v2 Announce Type: replace Abstract: An increasing number of NLP applications interact with large language models (LLMs) through black-box APIs, making prompt engineering critical for controlling model behavior.
By MohammadReza Davari, Utkarsh Garg, Weixin Cai, Eugene Belilovsky
arXiv:2511. 19829v2 Announce Type: replace Abstract: Most prompt-optimization methods refine a single static template, making them ineffective in complex and dynamic user scenarios.
By Ke Chen, Yifeng Wang, Hassan Almosapeeh, Haohan Wang
The paper introduces EvoPrompt, a framework for adapting vision‑language models to new tasks with limited data while preventing catastrophic forgetting. EvoPrompt uses a Modality‑Shared Prompt Projector to create hierarchical prompts and an evolutionary training strategy that separates low‑rank updates into directional and magnitude components, preserving learned semantic directions. Experiments show that EvoPrompt achieves state‑of‑the‑art few‑shot performance while maintaining the original zero‑shot capabilities of the pre‑trained models.
By Enming Zhang, Jiayang Li, Yanlong Wang, Yanru Wu, Zhenyu Liu, Yang Li
The paper investigates why prompt optimization works better for some tasks than others by decomposing reward variance into response variance and system‑prompt variance. It finds that optimization succeeds when system‑prompt variance dominates, and that adding more user prompts can actually reduce this variance, especially on heterogeneous datasets. To address this, the authors propose $p1$, a filtering method that selects a small set of high‑variance user prompts, which improves optimization on reasoning benchmarks and even allows a system prompt trained on just two AIME 24 prompts to generalize well.
By Zhaolin Gao (Sid), Yu (Sid), Wang, Bo Liu, Thorsten Joachims, Kiant\'e Brantley, Wen Sun
The paper investigates how different training strategies affect the prompt sensitivity of large language models. It reproduces and compares methods such as refined data construction and robustness objectives, finding that while robustness fine‑tuning improves over standard fine‑tuning and in‑context learning, the prompt gap remains large (40–57%). Notably, newer techniques like CoIN and PPCL often underperform a simple data‑construction approach that uses one template per batch, and diagnostics suggest that mixed‑template batches force the optimizer to reconcile conflicting updates rather than learn a prompt‑agnostic representation.
By Frederic Sadrieh, Michal \v{S}tef\'anik
arXiv:2608.29467v1 Announce Type: new
Abstract: Zero-shot anomaly detection (ZSAD) has gained significant attention for its practical value in industrial inspection. Recently, CLIP-based approaches h...
By Sisi Zhu, Changwei Yu, Renshuai Tao, Zhenliang Ni
arXiv:2511. 19829v3 Announce Type: replace Abstract: Prompt optimization has become a central mechanism for eliciting strong performance from LLMs, and recent work has made substantial progress by proposing diverse prompt evaluation metrics and optimization strategies.
By Ke Chen, Yifeng Wang, Hassan Almosapeeh, Haohan Wang