arXiv:2507. 09839v2 Announce Type: replace Abstract: An increasing number of NLP applications interact with large language models (LLMs) through black-box APIs, making prompt engineering critical for controlling model behavior.
arXiv:2511. 19829v3 Announce Type: replace Abstract: Prompt optimization has become a central mechanism for eliciting strong performance from LLMs, and recent work has made substantial progress by proposing diverse prompt evaluation metrics and optimization strategies.
arXiv:2511. 19829v2 Announce Type: replace Abstract: Most prompt-optimization methods refine a single static template, making them ineffective in complex and dynamic user scenarios.
The paper investigates how small lexical changes in prompts can cause large performance swings in large language models. Using a dataset of 132,000 prompt variants, the authors uncover a scaling law linking higher average task performance to lower variance and greater robustness. They identify domain-specific terminology and explicit action directives as key linguistic factors that stabilize prompts, and propose an automated Prompt-Refining Agent that reduces performance variance by 40.7% in code generation while maintaining or improving mean performance.
arXiv:2609.39927v1 Announce Type: new Abstract: Prompt optimization improves the performance of language-model systems on downstream tasks by refining their prompts. Classical methods evaluate prompt...
arXiv:2609.23716v1 Announce Type: cross Abstract: Textual-gradient methods automate prompt optimization through natural-language feedback, but their iterative updates can be unstable. We identify two...
arXiv:2609.15209v1 Announce Type: cross Abstract: Automatic prompt optimization (APO) improves language-model programs by revising prompts from task feedback, yet it typically holds its training data...
The paper introduces a fine-grained method called interactions to analyze prompt sensitivity in large language models (LLMs). By decomposing output scores into nonlinear interactions, the authors show that subtle prompt changes can destabilize these interactions even when overall outputs stay unchanged. They propose an Interaction-based Prompt Sensitivity (IPS) metric and use it to evaluate 50 open-source LLMs, finding that supervised fine‑tuning, larger model scales, dense architectures, and few‑shot learning all reduce prompt sensitivity, primarily by stabilizing low‑order interactions.
arXiv:2609.08572v1 Announce Type: new Abstract: Large language model (LLM)-based multi-agent systems (MAS) achieve strong performance by employing specialized multiple agents, yet their performance d...
The paper investigates how different training strategies affect the prompt sensitivity of large language models. It reproduces and compares methods such as refined data construction and robustness objectives, finding that while robustness fine‑tuning improves over standard fine‑tuning and in‑context learning, the prompt gap remains large (40–57%). Notably, newer techniques like CoIN and PPCL often underperform a simple data‑construction approach that uses one template per batch, and diagnostics suggest that mixed‑template batches force the optimizer to reconcile conflicting updates rather than learn a prompt‑agnostic representation.
arXiv:2506. 00400v4 Announce Type: replace-cross Abstract: LLM-based prompt optimization, which uses LLM-provided ``textual gradients'' (feedback) to refine prompts, has emerged as an effective method for automatic prompt engineering.
The paper presents a prompt-based method for minimal-edit grammatical error correction (GEC) that reduces overcorrection in large language models (LLMs). It introduces taxonomy-based instructions, batch prompting to regularize overcorrection, and LLM-assisted prompt optimization, achieving an $F_{0.5}$ score of 78.32 on BEA-2019 with Gemini 3.1-Pro. This approach narrows the performance gap to fine-tuned models while avoiding their infrastructure demands.
arXiv:2606. 11459v1 Announce Type: cross Abstract: Large Language Models are highly sensitive to prompt formulation, necessitating automatic prompt optimization to unlock their full potential.