Towards Data Science

Prompt Engineering Fails Quietly —  Prompt Regression Is Why

Small prompt changes can silently break critical behavior in production. This article introduces a practical framework to detect hidden regressions before users notice.

Towards Data Science
Jul 29

Prompt Engineering Is Solved—Prompt Management Isn’t

Prompt engineering helps you write better prompts—but it doesn’t help you change them safely. This article explores a common production failure where a simple variable rename breaks every live call, and introduces a lightweight static analysis tool that treats prompts like contracts, catching breaking changes before they ship.

By Emmimal P Alexander
arXiv AI
Aug 7

AISPA: User-Centric System Prompt Auditing for Large Language Model Applications

arXiv:2607. 28617v2 Announce Type: replace Abstract: System prompts are instructions configured by developers to govern the behaviors of foundation models in AI applications.

By Xiangning Lin, Shenzhe Zhu, Shu Yang, Zhenyu Zhang, Haoqian Zhang, Yipeng Zhao, Chengxuan Qian, Tianwei Wang, Ziheng Zhang, Zhenlong Yuan, Dingcheng Wang, Juncheng Wu, Yuan Si, Jiaxin Liu, Baolong Bi, Robert Mahari, Tobin South, Dazza Greenwood, Zexue He, Rishi Bommasani, Sophia Kazinnik, Andreas Haupt, Samuele Marro, Erik Brynjolfsson, Alex Pentland, Jiaxin Pei
arXiv AI
Jul 8

Prompt Coach: An Empirical Evaluation of an Agentic Tutor for Learning Prompt Engineering in Software Development

arXiv:2607. 06074v1 Announce Type: cross Abstract: Prompt engineering has emerged as a critical yet undertaught skill for software developers, one that traditional learning approaches are ill-equipped to support given its evolving, interactive, and context-dependent nature.

By Rohit Mehra, Kapil Singi, Vikrant Kaulgud, Vibhu Saujanya Sharma, Swapnajeet Gon Choudhury, Swati Sharma, Adam P. Burden, Majd Sakr
arXiv Machine Learning
5d ago

A Probe Direction Is a Property of Its Prompt

arXiv:2608. 13329v1 Announce Type: new Abstract: A model that behaves differently when it senses it is being tested would undermine the evaluations we rely on, so recent work has sought to read that sense directly from a model's activations.

By Valentin No\"el
arXiv AI
Jul 3

Prompt Coverage Adequacy

arXiv:2607. 02057v1 Announce Type: cross Abstract: In recent years, it has become increasingly evident that large language models (LLMs) and autonomous agents raise the level of abstraction in software development by shifting the focus from writing precise procedures to expressing intents and goals.

By Florian Tambon, Michael Konstantinou, Cedric Richter, Charles Chenouard, Mark Harman, Mike Papadakis