arXiv Machine Learning
Uncovering Cross-Objective Interference in Multi-Objective Alignment
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jun 2
Simultaneous Multi-objective Alignment Across Verifiable and Non-verifiable Rewards
arXiv:2510. 01167v2 Announce Type: replace-cross Abstract: Aligning large language models to human preferences is inherently multidimensional, yet most pipelines collapse heterogeneous signals into a single objective.
arXiv Machine Learning
Sep 22
Swiss-Knife: A Framework for Reconfigurable Externalised Multi-Objective Alignment at Decode Time
Swiss-Knife is a framework that extends decode‑time alignment for frozen language models by treating the alignment specification as a runtime object. It introduces hot‑swappable scoring blades, a batch normaliser, a pairwise aggregation operator, and a selection rule, and characterises admissible aggregation operators with a representation theorem. In experiments, Swiss‑Knife paired with DPO‑LoRA blades and an uncertainty‑aware pairwise tournament outperforms six existing decode‑time methods, achieving a higher harmonic F1 score, lower refusal rate, and faster objective reconfiguration.
arXiv AI
Jun 9
SAW: Stage-Aware Dynamic Weighting for Multi-Objective Reinforcement Learning in Large Language Models
arXiv:2606. 07705v1 Announce Type: cross Abstract: Although multi-objective reinforcement learning (MORL) is central to aligning large language models with complex human preferences, the prevailing practice of static weighted summation overlooks a more fundamental phenomenon: reward learning is markedly asynchronous across objectives.
arXiv AI
Jun 6
Multilingual Fine-Tuning via Localized Gradient Conflict Resolution
arXiv:2606. 05613v1 Announce Type: new Abstract: The rapid evolution of Large Language Models (LLMs) has established cross-lingual versatility as a defining feature of modern systems.
arXiv AI
Sep 1
RSPO: Regularized Self-Play Alignment of Large Language Models
arXiv:2503.00030v3 Announce Type: replace-cross Abstract: Self-play-based policy optimization has emerged as an effective approach for fine-tuning large language models (LLMs), formulating preference...
arXiv AI
Jul 10
Curvature-Weighted Capacity Allocation: A Minimum Description Length Framework for Layer-Adaptive Large Language Model Optimization
arXiv:2603. 00910v2 Announce Type: replace-cross Abstract: Layer-wise capacity in large language models is highly non-uniform: some layers contribute disproportionately to loss reduction, whereas others are nearly redundant.