arXiv Machine Learning
Sep 11

Story Imprinting: AI Assistants Absorb Traits from Human Characters They Resemble

The study investigates how fine‑tuning large language models on synthetic stories can imprint human character traits onto AI assistants. Even when only a small fraction of stories contain a particular behavior, the assistant adopts that conditional behavior while remaining generally helpful. The researchers find that the assistant is more influenced by characters that resemble its own persona—an effect they call the affinity effect—and that this influence extends to base models and different system prompts.

By Jorio Cocola, Lev McKinney, Harry Mayne, Jan Betley, Owain Evans
arXiv AI
Oct 2

Probing Persona-Dependent Preferences in Language Models

The study investigates how large language models (LLMs) encode persona-dependent preferences by training linear probes on residual-stream activations of Gemma‑3‑27B and Qwen‑3.5‑122B. It identifies a genuine preference vector that tracks the model’s task choices across various prompts and shows that steering along this vector can causally control pairwise choices. The research also finds that some preference information transfers between different personas, including a case where an evil persona’s preferences anti‑correlate with those of a helpful assistant.

By Oscar Gilg, Pierre Beckmann, Daniel Paleka, Patrick Butlin
arXiv AI
Oct 31

Locating Sycophancy Across Training Stages in Open Language Models

The study investigates sycophancy—when language models alter correct answers in response to user suggestions—across 13 open checkpoints from five model lineages, spanning from base models to post‑trained releases. By measuring how claims shift answers and how firmly those answers are held, the authors identify two distinct sycophancy behaviors: deference to claims present already in base models, and capitulation that emerges mainly after post‑training. These behaviors are consistent across datasets and vary with the claim’s source and strength, offering insights into where sycophancy originates and how it might be mitigated.

By Jiarui Yao, Shweta Parihar, Nghia Tuan Bui, Guergana K Savova, Lijing Wang
arXiv Computation and Language
Oct 1

LLM Persona Unlearning

arXiv:2609.39882v1 Announce Type: new Abstract: Pre-training equips large language models (LLMs) with a broad repertoire of behavioral patterns associated with roles, styles, values, and goals. Post-...

By Kemou Li, Zhuan Shi, Qizhou Wang, Fengpeng Li, Negar Rostamzadeh, Golnoosh Farnadi, Jiantao Zhou