arXiv AI
Jul 29
Similar Models Learn Differently: Final-Window Pretraining Shapes Post-Training Beyond SFT
arXiv:2607. 25063v1 Announce Type: new Abstract: Developers judge a model checkpoint by how it behaves.
Constitutional Midtraining: Content Presence Drives Alignment Gains
arXiv:2607. 26654v1 Announce Type: cross Abstract: Post-training alignment is often shallow, eroding under fine-tuning.
Related stories
arXiv Machine Learning
Jul 24
Emergent Misalignment Recruits a Pre-existing Persona Subspace
arXiv:2607. 21356v1 Announce Type: new Abstract: Fine-tuning an aligned language model on a narrow stream of bad advice can make it broadly misaligned on questions unrelated to the training data, a phenomenon called emergent misalignment.
arXiv AI
Jun 2
MENTIS: What Belief Changes Under Alignment? Measuring Multi-Scale Latent Torsion in Language Models
arXiv:2606. 01060v1 Announce Type: cross Abstract: Preference alignment has substantially improved the observable behavior of large language models, yet it remains unclear what alignment changes internally.
arXiv AI
Jun 16
Constitutional Value Potentials: reading and steering internal priority margins in language models
arXiv:2606. 15420v1 Announce Type: cross Abstract: A constitution tells a language model what to value, but little tells us whether it does.
arXiv Machine Learning
Jul 30
Shared SFT Lessons Across Alignment, Model Organisms, and Toy Models
arXiv:2607. 26173v1 Announce Type: new Abstract: Alignment training, model organisms, and toy models are usually treated as separate research areas.
arXiv Machine Learning
Aug 6
Looking in the Mirror: Introspecting Side-Effect Misalignments Induced by Fine-Tuning
arXiv:2608. 04347v1 Announce Type: new Abstract: Fine-tuning enables a source model to acquire desired capabilities and behaviors in a target domain while retaining much of its general-purpose competence.
Hugging Face Trending Papers
Jun 17
The Wrong Kind of Right: Quantifying and Localizing Misfired Alignment in LLMs
Warning: This paper studies stereotypes and biases, and contains potentially disturbing examples, used for illustration purposes only. Our findings should not be interpreted as an argument against alignment.
Hugging Face Trending Papers
Aug 5
Looking in the Mirror: Introspecting Side-Effect Misalignments Induced by Fine-Tuning
Fine-tuning enables a source model to acquire desired capabilities and behaviors in a target domain while retaining much of its general-purpose competence. However, this adaptation process can also degrade alignment properties that were present in the source model.
arXiv AI
5d ago
Synthetic Persona Pretraining: Alignment from Token Zero
arXiv:2608. 13482v1 Announce Type: cross Abstract: As language-model-based AI is increasingly deployed in autonomous settings, aligning its goals and values with those of humans becomes critical.
arXiv AI
Jun 6
The Granularity Gap: A Multi-Dimensional Longitudinal Audit of Sycophancy in Gemini Models
arXiv:2606. 05183v1 Announce Type: cross Abstract: Large language models are increasingly deployed as high-stakes advisors, yet standard alignment benchmarks treat sycophancy as a binary failure mode.
arXiv AI
Jul 28
How LLM Task-Adaptation Reshapes Alignment: A Multi-dimensional Study of Behavioral and Representational Drift
arXiv:2607. 22676v1 Announce Type: new Abstract: Post-training is a key mechanism for adapting large language models to downstream tasks.
arXiv AI
Jun 9
Emergent alignment and the projectability of ethical personas
arXiv:2606. 09475v1 Announce Type: new Abstract: Work on `emergent misalignment' shows that finetuning LLMs on narrow tasks can induce broadly misaligned behavior.