arXiv AI
Language Chain in Alignment: Cross-lingual Ranking Preference Optimization
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
5d ago
Language Chain in Alignment: Cross-Lingual Ranking Preference Optimization
arXiv:2608.23149v1 Announce Type: cross Abstract: The alignment of Large Language Models heavily relies on English-centric high-quality preference data, which often leads to suboptimal performance in...
arXiv AI
Jun 30
Aligning Language Model Benchmarks with Pairwise Preferences
arXiv:2602. 02898v3 Announce Type: replace Abstract: Language model benchmarks are pervasive and computationally-efficient proxies for real-world performance.
Hugging Face Trending Papers
5d ago
Preference Data Selection for Mitigating the Alignment Tax in Large Language Models
Aligning large language models to human preferences is crucial for real-world deployment but frequently incurs an alignment tax, leading to the catastrophic forgetting of pre-trained general capabilit...
arXiv AI
Jun 11
Toward Preference-aligned Large Language Models via Residual-based Model Steering
arXiv:2509. 23982v2 Announce Type: replace-cross Abstract: Preference alignment is a critical step in making Large Language Models (LLMs) useful and aligned with (human) preferences.
arXiv Machine Learning
Jun 30
REAR: Test-time Preference Realignment through Reward Decomposition
arXiv:2606. 30339v1 Announce Type: cross Abstract: Aligning large language models (LLMs) with diverse user preferences is a critical yet challenging task.
arXiv AI
4d ago
Preference Data Selection for Mitigating the Alignment Tax in Large Language Models
The paper introduces BALIGN, a balanced data selection strategy designed to reduce catastrophic forgetting—referred to as the alignment tax—in large language models during preference-based alignment. By analyzing preference optimization gradients, the authors identify three data-centric features that influence parameter drift: the reference model's log-probability margin, token length differences between chosen and rejected responses, and TF‑IDF similarity to general capability corpora. BALIGN aggregates these features into a composite risk score to filter out high-risk preference samples, thereby preserving foundational capabilities while maintaining alignment gains with minimal computational overhead.