arXiv AI
Optimal Design for Active Preference Learning with Biased LLM Judges
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jul 29
Less Data, Better Alignment: Data-Centric Multi-Evaluator Agreement for Preference Optimization
arXiv:2607. 25136v1 Announce Type: new Abstract: Research on preference optimization often varies the training objective while holding the data fixed.
arXiv AI
Jun 2
Margin Adaptive DPO: Leveraging Reward Model for Granular Control in Preference Optimization
arXiv:2510. 05342v2 Announce Type: replace-cross Abstract: Direct Preference Optimization (DPO) has emerged as a simple and effective method for aligning large language models.
arXiv Computation and Language
Aug 25
Length-Controlled Margin-Based Preference Optimization without Reference Model
arXiv:2502.14643v3 Announce Type: replace Abstract: Direct Preference Optimization (DPO) is a widely adopted offline algorithm for preference-based reinforcement learning from human feedback (RLHF),...
arXiv AI
4d ago
DIAL: Position-Debiased LLM Judges with Adaptive Human Preference Calibration
The paper introduces DIAL, a framework that uses large language models (LLMs) as judges while mitigating position bias and aligning their judgments with human preferences. DIAL separates judge‑specific position effects, learns shared structure in debiased LLM preferences, and adaptively calibrates this structure toward human targets using limited human comparisons. Experiments on simulations and three human‑preference benchmarks show that DIAL remains robust to unbalanced response order, achieves strong human‑aligned rankings with few labels, and adapts when LLM information is imperfect, supported by a real‑data study of over 410K judgments from 21 LLM judges.
arXiv Computation and Language
Sep 1
Low-Resource Preference Adaptation of LLMs via Activation-Based Label Propagation
arXiv:2608.30902v1 Announce Type: new Abstract: Adapting large language models to user-specific preferences is often constrained by the cost of human annotation, making preference optimisation imprac...
arXiv AI
Jun 2
S-SPPO: Semantic-Calibrated Self-Play Preference Optimization
arXiv:2606. 01561v1 Announce Type: new Abstract: Aligning Large Language Models (LLMs) with human preferences is often formulated via Direct Preference Optimization (DPO).