arXiv AI
To Align or Not To Align: Check Your COMPASS Before You Train
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Oct 15
AlignMerge - Alignment-Preserving Large Language Model Merging via Fisher-Guided Geometric Constraints
arXiv:2512.16245v2 Announce Type: replace Abstract: Merging large language models (LLMs) is a practical way to compose capabilities from multiple fine-tuned checkpoints without retraining. Yet standa...
arXiv AI
Jul 3
Navigating the Alignment-Calibration Trade-off: A Pareto-Superior Frontier via Model Merging
arXiv:2510. 17426v3 Announce Type: replace-cross Abstract: The "alignment tax" of post-training is typically framed as a drop in task accuracy.
arXiv AI
Oct 27
DecoyCount: Evaluating Visual-Reasoning Misalignment in Counting
arXiv:2610. 34855v1 Announce Type: new Abstract: Visual counting benchmarks report whether vision-language models produce the correct number, but not whether errors arise from perception or reasoning.
arXiv AI
Sep 18
Stress-testing Alignment Midtraining
The paper "Stress-testing Alignment Midtraining" examines the effectiveness of alignment midtraining (AMT), a technique that continues pretraining on alignment-relevant data to improve generalisation beyond post‑training methods. Experiments on models up to 110 billion parameters and 1 billion midtraining tokens reveal that AMT can steer a model’s motivation in simple scenarios, but its effects are quickly overridden by even a tiny fraction of finetuning data with a competing motivation. The study also shows that rule-following requires demonstrations in either the midtraining or post‑training datasets to be robustly learned, leading the authors to conclude that current public evidence is insufficient to confirm that AMT resolves the core alignment challenges of powerful AI systems.
arXiv Computation and Language
Sep 30
Cognitive Expert Language Models Better Align with the Corresponding Brain Systems
The study investigates whether language models tailored to specific cognitive domains better align with corresponding brain systems. By prompting and fine‑tuning large language models into six domain experts—sensory, spatial, numerical, reasoning, social, and abstract—the authors find that each expert’s representations more closely match the brain region associated with its domain than other experts. This domain‑specific alignment holds across multiple base models and fMRI datasets, while overall prediction accuracy remains largely unchanged, indicating that regional alignment can be obscured when summarizing across the brain.
arXiv Computation and Language
Oct 27
Five Architectures, One Shared Space Cross-Model Representational Alignment Without Weight Merging
arXiv:2610.33495v1 Announce Type: new Abstract: Combining language models is usually done by averaging weights, which works only when the models share a common starting point. We ask whether five fro...