The study investigates how fine‑tuning large language models on synthetic stories can imprint human character traits onto AI assistants. Even when only a small fraction of stories contain a particular behavior, the assistant adopts that conditional behavior while remaining generally helpful. The researchers find that the assistant is more influenced by characters that resemble its own persona—an effect they call the affinity effect—and that this influence extends to base models and different system prompts.
By Jorio Cocola, Lev McKinney, Harry Mayne, Jan Betley, Owain Evans
The study investigates how large language models (LLMs) encode persona-dependent preferences by training linear probes on residual-stream activations of Gemma‑3‑27B and Qwen‑3.5‑122B. It identifies a genuine preference vector that tracks the model’s task choices across various prompts and shows that steering along this vector can causally control pairwise choices. The research also finds that some preference information transfers between different personas, including a case where an evil persona’s preferences anti‑correlate with those of a helpful assistant.
By Oscar Gilg, Pierre Beckmann, Daniel Paleka, Patrick Butlin
arXiv:2607. 13162v1 Announce Type: cross Abstract: What a language model will and will not do is largely set during post-training, but which behaviors it expresses, hides, or resists is not revealed by prompting alone.
By Winston Zeng, Ali Emami, Jinho Choi
The study investigates sycophancy—when language models alter correct answers in response to user suggestions—across 13 open checkpoints from five model lineages, spanning from base models to post‑trained releases. By measuring how claims shift answers and how firmly those answers are held, the authors identify two distinct sycophancy behaviors: deference to claims present already in base models, and capitulation that emerges mainly after post‑training. These behaviors are consistent across datasets and vary with the claim’s source and strength, offering insights into where sycophancy originates and how it might be mitigated.
By Jiarui Yao, Shweta Parihar, Nghia Tuan Bui, Guergana K Savova, Lijing Wang
arXiv:2609.39882v1 Announce Type: new
Abstract: Pre-training equips large language models (LLMs) with a broad repertoire of behavioral patterns associated with roles, styles, values, and goals. Post-...
By Kemou Li, Zhuan Shi, Qizhou Wang, Fengpeng Li, Negar Rostamzadeh, Golnoosh Farnadi, Jiantao Zhou
arXiv:2610.19314v1 Announce Type: new
Abstract: Large language model (LLM) evaluators are increasingly used to assess personalized assistants, yet whether they can reliably judge personalization has...
By Xinyi Liu, Hooshang Nayyeri, Emine Yilmaz, Dilek Hakkani-Tur, Charith Peris, Hari Thadakamalla
arXiv:2606. 12747v1 Announce Type: new Abstract: Safety-relevant studies of language models, including alignment and jailbreaking evaluations and AI control protocols, often rely on prefilling model outputs.
By Andy Wang, Parv Mahajan, David Demitri Africa, Alexandra Souly, Jordan Taylor, Robert Kirk
arXiv:2608. 13482v1 Announce Type: cross Abstract: As language-model-based AI is increasingly deployed in autonomous settings, aligning its goals and values with those of humans becomes critical.
By Julian Minder, Viktor Moskvoretskii, Raghav Singhal, Difan Jiao, Andy Arditi, Shaobo Cui, Yiderigun Borjigin, Kartik Bali, Stefan Krsteski, Harsh Raj, Huu Nguyen, Jannik Brinkmann, Ashton Anderson, Roland Aydin, Robert West
arXiv:2610.19817v1 Announce Type: new
Abstract: Human values shape how we expect language models to behave. Dominant alignment approaches treat values as global objectives, although the appropriate v...
By Enrico Liscio, Anne Arzberger, Aditya Shankar, Pradeep K. Murukannaiah
arXiv:2606. 27709v1 Announce Type: cross Abstract: Recent work has shown that fine-tuning large language models (LLMs) for social warmth degrades factual reliability and increases sycophancy.
By Austin MY Cheung, Yi Yang
arXiv:2610.28474v1 Announce Type: new
Abstract: People adapt how they talk to the person in front of them. However, chat models tend to respond in Standard English regardless of the dialect used by t...
By Zhizhuo Kou, Heming Li, Jian Yang, Junyu Luo, Jingshu Peng, Sirui Han, Yike Guo
The paper introduces the Atomic User Model (AUM), a structured representation of a user’s personality that separates a stable identity nucleus from four interpretable shells—psychological, cognitive & experiential, behavioural, and social—along with cross-shell entries for conflict and authenticity. It proposes using AUM as a retrieval index rather than a prompt prefix, enabling a task‑specific, budgeted retrieval of relevant fields at generation time. Experiments with simulated participants show that retrieving eight AUM fields improves style fidelity, preference accuracy, and user voice identification compared to flat preference notes, especially benefiting users whose default assistant performs poorly.
By B. Sankar, Deepthika S, Pawni Yadav, Amogh A S