arXiv Machine Learning July 14, 2026 By Hua Qu, Yifan Li, Xiaodong Yuan

Metadata-Free Meta-Reweighted Direct Preference Optimization under Noisy Preference Labels

Read the original on arXiv Machine Learning →

arXiv:2607. 09796v1 Announce Type: new Abstract: Direct Preference Optimization (DPO) has become an important method for aligning large language models (LLMs) with human preferences because it removes the need for explicit reward modeling and reinforcement learning optimization.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Jun 2

Margin Adaptive DPO: Leveraging Reward Model for Granular Control in Preference Optimization

arXiv:2510. 05342v2 Announce Type: replace-cross Abstract: Direct Preference Optimization (DPO) has emerged as a simple and effective method for aligning large language models.

By Hyung Gyu Rho

llms nlp reinforcement-learning

arXiv AI

Jun 30

Distributionally Robust Reinforcement Learning with Human Feedback

arXiv:2503. 00539v2 Announce Type: replace-cross Abstract: Reinforcement learning from human feedback (RLHF) has evolved to be one of the main methods for fine-tuning large language models (LLMs).

By Debmalya Mandal, Paulius Sasnauskas, Goran Radanovic

llms reinforcement-learning fine-tuning

arXiv AI

Aug 11

Learning Preference Adaptation for Large Language Model Personalization via Verbal Reinforcement Learning

arXiv:2608. 09507v1 Announce Type: cross Abstract: Natural language user preferences provide an interpretable interface for LLM personalization.

By Yuting Liu, Wei Wu, Jianzhe Zhao, Guibing Guo

llms rag agents reinforcement-learning

arXiv Machine Learning

Jul 7

Labeled-Data-Free Meta-Learning: Efficient Task Generation Using Pre-trained Models and Unlabeled Data

arXiv:2607. 02850v1 Announce Type: new Abstract: Meta-learning without labeled data is crucial for real-world applications, where obtaining labeled datasets can be expensive or restricted due to privacy concerns.

By Lei Sun, Yusuke Tanaka, Tomoharu Iwata

benchmarks safety

arXiv Machine Learning

Jul 30

Meta-Learned Reward Shaping for Reinforcement Learning from Human Feedback

arXiv:2607. 26094v1 Announce Type: new Abstract: Reinforcement Learning from Human Feedback (RLHF) is the standard approach for aligning large language models with human preferences, but its quality is limited by static, task-agnostic reward models.

By Yunpeng Chu

llms reinforcement-learning benchmarks safety

arXiv AI

1d ago

Leveraging Machine Unlearning for Cost-Efficient Preference Alignment

arXiv:2504. 06659v2 Announce Type: replace-cross Abstract: Despite advances in Preference Alignment (PA) for Large Language Models (LLMs), mainstream methods like reinforcement learning with human feedback face notable challenges.

By Xiaohua Feng, Yuyuan Li, Huwei Ji, Jiaming Zhang, Li Zhang, Tianyu Du, Chaochao Chen

llms reinforcement-learning safety

Related stories

Margin Adaptive DPO: Leveraging Reward Model for Granular Control in Preference Optimization

Distributionally Robust Reinforcement Learning with Human Feedback

Learning Preference Adaptation for Large Language Model Personalization via Verbal Reinforcement Learning

Labeled-Data-Free Meta-Learning: Efficient Task Generation Using Pre-trained Models and Unlabeled Data

Meta-Learned Reward Shaping for Reinforcement Learning from Human Feedback

Leveraging Machine Unlearning for Cost-Efficient Preference Alignment