← Back to all news
arXiv Machine Learning By Yasuaki Sumita, Tomoharu Iwata, Toshiyuki Tanaka

Controlling the Diversity-Quality Trade-Off of LLMs by Merging Models Before and After Alignment

Read the original on arXiv Machine Learning →

The paper proposes a depth‑aware FrankenMerging technique that blends lower layers of an aligned large language model (LLM) with higher layers of its base counterpart to balance diversity and quality without extra training. Experiments demonstrate that adjusting the layer split controls both lexical and semantic diversity, often surpassing the aligned model in semantic diversity and outperforming temperature sampling in most settings. The method is validated by an ablation that reverses the merge order, supporting the hypothesis about layer‑specific diversity contributions.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Computation and Language
Aug 28

Cross-Lingual Alignment Without Joint Training: Do Monolingual Language Models Converge on Universal Representations?

The study investigates whether monolingual language models, trained without joint multilingual objectives, develop cross-lingual alignment. By evaluating models such as Goldfish and independently built monolingual systems, the authors find that alignable representational geometry emerges across layers, strengthening with larger data, larger models, or closer linguistic proximity. A single Procrustes rotation on parallel sentences can map hidden states between models, and applying this rotation to a German model’s residuals swaps factual predictions to those of the donor English model, demonstrating functional transfer.

By Ej Zhou, Suchir Salhan, Catherine Arnett, Anna Korhonen
llmssafety
More like this →
arXiv AI
Aug 25

An Empirical Study on Preference Tuning Generalization and Diversity Under Domain Shift

The paper investigates how preference tuning—optimizing language models with explicit preference signals—behaves when applied to new domains. It systematically compares five alignment objectives and several adaptation strategies, such as target‑domain supervised fine‑tuning and pseudo‑labeling, across summarization, question‑answering helpfulness, and safety tasks. Results show that while pseudo‑labeling reduces domain‑shift degradation, it also causes mode collapse, highlighting a trade‑off between generalization and diversity.

By Constantinos Karouzos, Xingwei Tan, Nikolaos Aletras
llmsnlpfine-tuningsafety
More like this →