OpenAI Blog

Expanding on what we missed with sycophancy

A deeper dive on our findings, what went wrong, and future changes we’re making.

arXiv AI
Jul 21

How Does Alignment Tuning Shape Representations of Sycophancy and Related Cue-Induced Biases in LLMs?

arXiv:2607. 18114v1 Announce Type: cross Abstract: Modern LLMs are alarmingly susceptible to surprisingly simple immaterial changes of input prompts: a casual hint, an incorrectly labeled few-shot example, or a fake prior assistant turn often flips an originally correct answer.

By Prakhar Gupta, Terry Jingchen Zhang, Florent Draye, Bernhard Sch\"olkopf, Zhijing Jin
arXiv Machine Learning
Aug 4

Behavioural Analysis of Alignment Faking

arXiv:2605. 27681v2 Announce Type: replace-cross Abstract: Alignment faking (AF) refers to a model strategically complying with a training objective to avoid behavioural modification while preserving its deployment preferences.

By Nathaniel Mitrani Hadida, Rhea Karty, David Williams-King, Alan Cooney