Preference Tuning LLMs with Direct Preference Optimization Methods

Beyond Uniform Forgetting: A Study of Sequential Direct Preference Optimization Across Preference Settings

arXiv:2606. 19744v1 Announce Type: cross Abstract: Aligning language models with human preferences often requires optimising multiple behavioural objectives.

Boosting Direct Preference Optimization with Penalization

arXiv:2606. 12505v1 Announce Type: cross Abstract: Offline preference optimization has become a practical substitute for reinforcement learning from human feedback, but pairwise objectives such as Direct Preference Optimization (DPO) and its variants use only the chosen and rejected responses stored in a static dataset.

Optimizing your LLM in production

Direct Preference Optimization Beyond Chatbots

Make LLM Fine-tuning 2x faster with Unsloth and 🤗 TRL

Autoregressive Direct Preference Optimization

arXiv:2602. 09533v2 Announce Type: replace Abstract: Direct preference optimization (DPO) has emerged as a promising approach for aligning large language models (LLMs) with human preferences.