arXiv AI
Jul 29
Similar Models Learn Differently: Final-Window Pretraining Shapes Post-Training Beyond SFT
arXiv:2607. 25063v1 Announce Type: new Abstract: Developers judge a model checkpoint by how it behaves.
Constitutional Midtraining: Content Presence Drives Alignment Gains
arXiv:2607. 26654v1 Announce Type: cross Abstract: Post-training alignment is often shallow, eroding under fine-tuning.
Related stories
arXiv AI
Sep 18
Stress-testing Alignment Midtraining
The paper "Stress-testing Alignment Midtraining" examines the effectiveness of alignment midtraining (AMT), a technique that continues pretraining on alignment-relevant data to improve generalisation beyond post‑training methods. Experiments on models up to 110 billion parameters and 1 billion midtraining tokens reveal that AMT can steer a model’s motivation in simple scenarios, but its effects are quickly overridden by even a tiny fraction of finetuning data with a competing motivation. The study also shows that rule-following requires demonstrations in either the midtraining or post‑training datasets to be robustly learned, leading the authors to conclude that current public evidence is insufficient to confirm that AMT resolves the core alignment challenges of powerful AI systems.
arXiv Machine Learning
Jul 24
Emergent Misalignment Recruits a Pre-existing Persona Subspace
arXiv:2607. 21356v1 Announce Type: new Abstract: Fine-tuning an aligned language model on a narrow stream of bad advice can make it broadly misaligned on questions unrelated to the training data, a phenomenon called emergent misalignment.
arXiv AI
Jun 2
MENTIS: What Belief Changes Under Alignment? Measuring Multi-Scale Latent Torsion in Language Models
arXiv:2606. 01060v1 Announce Type: cross Abstract: Preference alignment has substantially improved the observable behavior of large language models, yet it remains unclear what alignment changes internally.
arXiv AI
Jun 16
Constitutional Value Potentials: reading and steering internal priority margins in language models
arXiv:2606. 15420v1 Announce Type: cross Abstract: A constitution tells a language model what to value, but little tells us whether it does.
arXiv Machine Learning
Jul 30
Shared SFT Lessons Across Alignment, Model Organisms, and Toy Models
arXiv:2607. 26173v1 Announce Type: new Abstract: Alignment training, model organisms, and toy models are usually treated as separate research areas.
arXiv Computation and Language
Sep 2
Joint Training Is Not Enough: Conditioned Cross-Granularity Training for Multimodal Document Understanding
arXiv:2609.00756v1 Announce Type: new Abstract: The Mutual Reinforcement Effect (MRE) asks whether a fine, span-level and a coarse, document-level task help each other when one model handles both. We...
arXiv Machine Learning
Aug 6
Looking in the Mirror: Introspecting Side-Effect Misalignments Induced by Fine-Tuning
arXiv:2608. 04347v1 Announce Type: new Abstract: Fine-tuning enables a source model to acquire desired capabilities and behaviors in a target domain while retaining much of its general-purpose competence.
Hugging Face Trending Papers
Jun 17
The Wrong Kind of Right: Quantifying and Localizing Misfired Alignment in LLMs
Warning: This paper studies stereotypes and biases, and contains potentially disturbing examples, used for illustration purposes only. Our findings should not be interpreted as an argument against alignment.
arXiv AI
4d ago
Safer Content or Firmer Refusals? A Hybrid Perturbation Defense for Alignment under Harmful Fine-tuning
arXiv:2609.36862v1 Announce Type: cross Abstract: Fine-tuning-as-a-service lets users adapt a safety-aligned language model to their own data, but it also creates a harmful fine-tuning attack surface...
Hugging Face Trending Papers
Aug 5
Looking in the Mirror: Introspecting Side-Effect Misalignments Induced by Fine-Tuning
Fine-tuning enables a source model to acquire desired capabilities and behaviors in a target domain while retaining much of its general-purpose competence. However, this adaptation process can also degrade alignment properties that were present in the source model.
arXiv AI
4d ago
Constitutional adapters: Inference-time interventions for misalignment and misuse
arXiv:2609.36657v1 Announce Type: cross Abstract: Training models to act in accordance with an explicitly defined set of principles, or "constitution," has shown promise as a robust and transparent m...