Continuous-Utility Direct Preference Optimization (CU‑DPO) replaces binary preference labels with continuous scores to better capture fine‑grained reasoning quality. The framework aligns language models to multiple prompt‑based cognitive strategies, proving a θ(K log K) sample‑complexity improvement and convergence to an entropy‑regularized utility‑maximizing policy. A two‑stage pipeline—strategy selection via best‑vs‑all comparisons and execution refinement using margin‑stratified pairs—yields significant gains on mathematical reasoning, code generation, and causal reasoning benchmarks.
arXiv:2601.03822v2 Announce Type: replace Abstract: Large language models (LLMs) can achieve strong reasoning performance with sufficient computation, but they do not inherently know how much computa...
arXiv:2607. 06987v1 Announce Type: new Abstract: Reinforcement learning (RL) has become the standard paradigm for enhancing the complex reasoning capabilities of large language models (LLMs).
arXiv:2603. 25184v2 Announce Type: replace-cross Abstract: Reinforcement learning (RL) has become essential for post-training large language models (LLMs) in reasoning tasks.
arXiv:2606. 25832v1 Announce Type: new Abstract: Achieving strong optimization generalization across diverse optimization problems while requiring limited training resources remains a challenging problem for optimization-oriented large language models (LLMs).
The paper introduces Graph Direct Preference Optimization (GraphDPO), a generalization of Direct Preference Optimization that uses directed acyclic graphs to capture richer preference structures from multiple rollouts per prompt. GraphDPO encodes dominance relations as edges and optimizes a Plackett–Luce-inspired objective that aggregates supervision over dominated neighborhoods, preserving transitivity and handling ties or incomparable responses via an equivalence-class construction. Experiments on reasoning and program synthesis tasks show that this graph-structured approach outperforms traditional pairwise and listwise alignment objectives while maintaining linear per‑prompt complexity.
arXiv:2607. 11089v1 Announce Type: new Abstract: Large Language Models (LLMs) have achieved remarkable success in complex reasoning tasks through Chain-of-Thought (CoT) prompting.
arXiv:2606. 01281v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as a powerful paradigm for enhancing the reasoning capabilities of large language models (LLMs).
arXiv:2604. 20140v2 Announce Type: replace Abstract: Direct Preference Optimization (DPO) is an effective framework for aligning large language models with human preferences, but it struggles with complex reasoning tasks.
Achieving strong optimization generalization across diverse optimization problems while requiring limited training resources remains a challenging problem for optimization-oriented large language models (LLMs). Existing approaches typically rely on large-scale supervised datasets, costly reasoning annotations, and expensive intermediate step verification, resulting in substantial training overhead.
arXiv:2604.02341v2 Announce Type: replace-cross Abstract: Mathematical reasoning in large language models has improved substantially with reinforcement learning using verifiable rewards, where final...
arXiv:2508. 09883v2 Announce Type: replace-cross Abstract: Large language models (LLMs) demonstrate remarkable reasoning capabilities in tasks such as algorithmic coding and mathematical problem-solving.
Reinforcement learning with verifiable rewards (RLVR) improves the reasoning capabilities of large language models, but prompt groups with identical rollout rewards consume generation budget without effective learning signals. Pre-rollout prompt selection can reduce this waste by screening prompts before rollout generation.