arXiv AI By Zixin Ding, Junyuan Hong, Zhan Shi, Jiachen T. Wang, Zinan Lin, Li Yin, Meng Liu, Zhangyang Wang, Yuxin Chen

Scaling Textual Gradients via Sampling-Based Momentum

Read the original on arXiv AI →

arXiv:2506. 00400v4 Announce Type: replace-cross Abstract: LLM-based prompt optimization, which uses LLM-provided ``textual gradients'' (feedback) to refine prompts, has emerged as an effective method for automatic prompt engineering.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Sep 2

Prompt-Robust Language Models: Which Training Strategies Work?

The paper investigates how different training strategies affect the prompt sensitivity of large language models. It reproduces and compares methods such as refined data construction and robustness objectives, finding that while robustness fine‑tuning improves over standard fine‑tuning and in‑context learning, the prompt gap remains large (40–57%). Notably, newer techniques like CoIN and PPCL often underperform a simple data‑construction approach that uses one template per batch, and diagnostics suggest that mixed‑template batches force the optimizer to reconcile conflicting updates rather than learn a prompt‑agnostic representation.

By Frederic Sadrieh, Michal \v{S}tef\'anik