arXiv Machine Learning August 27, 2026 By Dongyue Li, Ziniu Zhang, Lu Wang, Hongyang R. Zhang

Learning Mixtures of Plackett-Luce Models for Multi-Objective Alignment

Read the original on arXiv Machine Learning →

The paper introduces MoPLEx, an expectation‑maximization algorithm for learning mixtures of Plackett‑Luce models from multi‑way ranking data. It augments rankings with synthetic responses from a base language model and uses a gradient‑based estimation to reduce inference cost, enabling efficient fitting of large‑scale models. Experiments show the method achieves low probability estimation error and improves clustering and ranking accuracy by 43.7% and 15.2% over baselines.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Sep 1

MoPLEx: Estimating Plackett-Luce Mixture Models for Multi-Objective Alignment

arXiv:2608.25200v2 Announce Type: replace-cross Abstract: We study learning a mixture of $k$ Plackett-Luce models from multi-way ranking responses from annotators that may represent heterogeneous und...

By Dongyue Li, Ziniu Zhang, Lu Wang, Hongyang R. Zhang

rag safety

arXiv Machine Learning

Jun 9

Nonparametric LLM Evaluation from Preference Data

arXiv:2601. 21816v2 Announce Type: replace Abstract: Evaluating the performance of large language models (LLMs) from human preference data is crucial for obtaining LLM leaderboards.

By Dennis Frauen, Athiya Deviyani, Mihaela van der Schaar, Stefan Feuerriegel

llms benchmarks

arXiv AI

Aug 18

Leveraging Machine Unlearning for Cost-Efficient Preference Alignment

arXiv:2504. 06659v2 Announce Type: replace-cross Abstract: Despite advances in Preference Alignment (PA) for Large Language Models (LLMs), mainstream methods like reinforcement learning with human feedback face notable challenges.

By Xiaohua Feng, Yuyuan Li, Huwei Ji, Jiaming Zhang, Li Zhang, Tianyu Du, Chaochao Chen

llms reinforcement-learning safety

Hugging Face Trending Papers

Aug 25

Preference Data Selection for Mitigating the Alignment Tax in Large Language Models

Aligning large language models to human preferences is crucial for real-world deployment but frequently incurs an alignment tax, leading to the catastrophic forgetting of pre-trained general capabilit...

llms safety

arXiv AI

Jun 30

Aligning Language Model Benchmarks with Pairwise Preferences

arXiv:2602. 02898v3 Announce Type: replace Abstract: Language model benchmarks are pervasive and computationally-efficient proxies for real-world performance.

By Marco Gutierrez, Xinyi Leng, Hannah Cyberey, Jonathan Richard Schwarz, Ahmed Alaa, Thomas Hartvigsen

llms benchmarks safety

Hugging Face Trending Papers

Jul 1

Multi-Objective Exploration and Preference Optimization via Mutual Information

Aligning large language models with diverse and heterogeneous human values requires multi-objective alignment methods to effectively trade off conflicting preference dimensions. Current methods achieve this trade-off by training policies conditioned on preference vectors and leveraging online direct preference optimization.

llms safety

One email a morning, machine-written

Related stories

MoPLEx: Estimating Plackett-Luce Mixture Models for Multi-Objective Alignment

Nonparametric LLM Evaluation from Preference Data

Leveraging Machine Unlearning for Cost-Efficient Preference Alignment

Preference Data Selection for Mitigating the Alignment Tax in Large Language Models

Aligning Language Model Benchmarks with Pairwise Preferences

Multi-Objective Exploration and Preference Optimization via Mutual Information