arXiv AI By Truong Nguyen, Tien-Phat Nguyen, Linh Ngo Van, Duy Minh Ho Nguyen, Khoa Doan, Trung Le

TokenRatio: Principled Token-Level Preference Optimization via Ratio Matching

Read the original on arXiv AI →

arXiv:2605. 12288v3 Announce Type: replace-cross Abstract: Direct Preference Optimization (DPO) is a widely used RL-free method for aligning language models from pairwise preferences, but it models preferences over full sequences even though generation is driven by per-token decisions.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.