Hugging Face Blog
Direct Preference Optimization Beyond Chatbots
Read the original on Hugging Face Blog →The Flow has not summarised this story yet — read it at Hugging Face Blog.
Related stories
arXiv Machine Learning
Jun 15
Direct Preference Optimization for Chatbot Fine-Tuning: An Empirical Study
arXiv:2606. 12881v2 Announce Type: replace-cross Abstract: We present an approach to fine-tuning large language models using Direct Preference Optimization (DPO), a reinforcement learning technique.
arXiv Machine Learning
Jul 10
Dropping Just a Handful of Preferences Can Change Top Large Language Model Rankings
arXiv:2508. 11847v4 Announce Type: replace-cross Abstract: We propose a method for evaluating the robustness of widely used LLM ranking systems -- variants of a Bradley--Terry model -- to dropping a worst-case very small fraction of preference data.
arXiv AI
Jul 28
Steerable Chatbots: Exploring Personalization Control Interfaces via LLM Activation Steering
arXiv:2505. 04260v3 Announce Type: replace-cross Abstract: Personalizing LLM responses typically requires users to articulate their preferences through prompting, which can be burdensome at cold start and difficult to articulate in natural language.
arXiv AI
Jun 10
A Comprehensive Survey of Direct Preference Optimization: Datasets, Theories, Variants, and Applications
arXiv:2410. 15595v4 Announce Type: replace Abstract: With the rapid advancement of large language models (LLMs), aligning policy models with human preferences has become increasingly critical.
arXiv AI
Jul 1
Beyond expert users: agents should help users construct preferences, not just elicit them
arXiv:2606. 30863v1 Announce Type: new Abstract: Agents typically assume an expert user -- one with well-formed preferences about what they want -- and default to clarifying questions whenever the task is underspecified.
arXiv AI
Jun 6
Statistical Priors for Implicit Preferences: Decoupling Skill Selection as a Local Harness in Personal Agents
arXiv:2606. 05828v1 Announce Type: new Abstract: As Large Language Model (LLM) capabilities advance, locally deployed personal agents relying on API-based remote models and external skills have emerged as a novel paradigm.