arXiv Computation and Language

LLM Persona Unlearning

arXiv AI
Aug 14

Synthetic Persona Pretraining: Alignment from Token Zero

arXiv:2608. 13482v1 Announce Type: cross Abstract: As language-model-based AI is increasingly deployed in autonomous settings, aligning its goals and values with those of humans becomes critical.

By Julian Minder, Viktor Moskvoretskii, Raghav Singhal, Difan Jiao, Andy Arditi, Shaobo Cui, Yiderigun Borjigin, Kartik Bali, Stefan Krsteski, Harsh Raj, Huu Nguyen, Jannik Brinkmann, Ashton Anderson, Roland Aydin, Robert West
arXiv AI
Oct 31

Persona Learning from Feedback for Personalized AI Assistants

The paper introduces Persona Learning from Feedback (PLF), a framework that creates reusable, preference‑grounded assistant personas as LoRA adapters on a frozen backbone. During training, a frozen self‑teacher provides token‑level supervision conditioned on persona specifications and response‑specific feedback, while a feedback gate focuses updates on responses needing improvement. At deployment, a supervised router selects an appropriate adapter or composition based on the request or dialogue, enabling personalized AI assistants without user‑specific fine‑tuning and improving preference fit over existing persona prompting and distillation methods.

By Yiran Shen, Yu Xia, Liuyi Yao, Prithviraj Ammanabrolu
arXiv AI
Sep 10

Training-Free Task Vectors for LLM Behavioral Control

The paper introduces Training-Free Task Vectors (TFTVs), a method for computing task-vector-like directions in large language models without fine‑tuning. TFTVs map activation steering vectors to rank‑one weight‑space edits using only forward‑pass statistics, enabling arithmetic operations such as learning, forgetting, and composing edits. Experiments show that TFTVs consistently amplify, suppress, and combine target behaviors while preserving general knowledge, outperforming other editing and steering baselines.

By Gabriel J. Perin, Lucas Boscaini, Andr\'e Araujo, Nina S. T. Hirata