arXiv AI
Steerable Pluralistic Policy Alignment from Anonymized Crowd Feedback
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 1
Personalized Group Relative Policy Optimization for Heterogenous Preference Alignment
The paper introduces Personalized Group Relative Policy Optimization (P‑GRPO), a new alignment framework for large language models that separates advantage estimation from immediate batch statistics. By normalizing advantages using preference‑group‑specific reward histories instead of the concurrent generation group, P‑GRPO maintains contrastive signals for distinct user preferences. Experiments across various tasks show that P‑GRPO converges faster and yields higher rewards than standard GRPO, improving alignment with heterogeneous human preferences while preserving general capabilities.
arXiv Machine Learning
Jun 8
Adaptive Pluralistic Alignment: A pipeline for dynamic artificial democracy
arXiv:2605. 01642v2 Announce Type: replace Abstract: Prevailing alignment methods target a fixed set of preferences and therefore risk forcing value lock-in as societal norms evolve over time.
arXiv AI
Sep 16
MiCRo: Mixture Modeling and Context-aware Routing for Personalized Preference Learning
MiCRo is a two‑stage framework that improves personalized preference learning for large language models. It first uses a context‑aware mixture model to capture diverse human preferences from large binary preference datasets, then applies an online routing strategy to dynamically adjust mixture weights based on context, reducing ambiguity. Experiments on multiple datasets show that MiCRo captures diverse preferences and enhances downstream personalization.
arXiv Machine Learning
Aug 4
PB$^2$: Preference Space Exploration via Population-Based Methods in Preference-Based Reinforcement Learning
arXiv:2506. 13741v2 Announce Type: replace-cross Abstract: Preference-based reinforcement learning (PbRL) has emerged as a promising approach for learning behaviors from human feedback without predefined reward functions.
arXiv AI
Jul 7
Adaptive Margin RLHF via Preference over Preferences
arXiv:2509. 22851v4 Announce Type: replace-cross Abstract: Margin-based optimization is fundamental to improving generalization and robustness in classification tasks.
arXiv AI
Jun 11
Toward Preference-aligned Large Language Models via Residual-based Model Steering
arXiv:2509. 23982v2 Announce Type: replace-cross Abstract: Preference alignment is a critical step in making Large Language Models (LLMs) useful and aligned with (human) preferences.