arXiv Machine Learning October 8, 2026 By Rabimba Karanjai, Qun Gu, Hemanth Hegadehalli Madhavarao, Wenhuan Sun, Xiaojiao Yu, Suryabhan Singh Hada, Libin N. George, Uma Kona, Richard Williamson, Linsey Pang, Prakhar Mehrotra

CM-DPO: Constraint-Margin Direct Preference Optimization for LLM Planning

Read the original on arXiv Machine Learning →

CM-DPO introduces a Constraint-Margin Direct Preference Optimization method that replaces the binary preference signal of traditional DPO with a continuous margin based on a deterministic symbolic verifier, scaled by violation severity. Hard and soft constraints are separated via a lexicographic objective to prevent hard constraints from being traded off against preferences. Using procedurally generated constraint profiles and minimal-edit distillation within the SynPlan-R framework, an 8B model fine‑tuned with CM‑DPO achieves high pass and solve rates on several planning benchmarks, matching multi‑agent systems with lower latency and outperforming GPT‑4o on unseen tasks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jun 4

Beyond Objective Equivalence: Constraint Injection for LLM-Based Optimization Modeling on Vehicle Routing Problems

arXiv:2606. 04816v1 Announce Type: new Abstract: Large language models (LLMs) increasingly translate natural-language optimization problems into executable solver code.

By Xizi Luo, Changhong He, Dongdong Geng, Chenggong Shi, Yu Mei

llms benchmarks

arXiv AI

Jun 2

Margin Adaptive DPO: Leveraging Reward Model for Granular Control in Preference Optimization

arXiv:2510. 05342v2 Announce Type: replace-cross Abstract: Direct Preference Optimization (DPO) has emerged as a simple and effective method for aligning large language models.

By Hyung Gyu Rho

llms nlp reinforcement-learning

arXiv AI

Jul 7

CARL: Constraint-Aware Reinforcement Learning for Planning with LLMs

arXiv:2607. 04854v1 Announce Type: new Abstract: Despite their strong reasoning capabilities and extensive world knowledge, Large Language Models (LLMs) frequently generate plans that violate task constraints, undermining their reliability in real-world applications.

By Qiuyi Qi, Jinjian Zhang, Mutian Bao, Tian Liang, Guocong Li, Dongnan Liu, Wei Zhou, Jie Liu, Ming Kong, Linjian Mo, Feng Zhang, Qiang Zhu

llms reinforcement-learning fine-tuning benchmarks

arXiv AI

Aug 18

CAPO: Constraint-Aware Prompt Optimization for LLM Agents

arXiv:2608. 16068v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed as agents that rely on system prompts to use tools and complete tasks.

By Victor Ye Dong, Reid Pryzant, Yi Liu, Jian Jiao

llms agents benchmarks safety

arXiv AI

Jul 20

Hard Rules, Soft Preferences: Bridging Reasoning, Learning, and Optimization for Personalized Packing Checklist Generation

arXiv:2607. 15562v1 Announce Type: cross Abstract: Packing for air travel is recurring and error-prone: the checklist must be personal and context-aware, yet feasible under safety rules, item dependencies, and luggage limits.

By Himel Dev, Madhusudan Basak, Tanmoy Sen, Paromita Shome, Bashima Islam

llms safety

Hugging Face Trending Papers

Jul 6

CARL: Constraint-Aware Reinforcement Learning for Planning with LLMs

Despite their strong reasoning capabilities and extensive world knowledge, Large Language Models (LLMs) frequently generate plans that violate task constraints, undermining their reliability in real-world applications. This deficiency arises from a lack of systematic mechanisms to incorporate constraint information during the generation process.

llms reinforcement-learning fine-tuning benchmarks

One email a morning, machine-written

Related stories

Beyond Objective Equivalence: Constraint Injection for LLM-Based Optimization Modeling on Vehicle Routing Problems

Margin Adaptive DPO: Leveraging Reward Model for Granular Control in Preference Optimization

CARL: Constraint-Aware Reinforcement Learning for Planning with LLMs

CAPO: Constraint-Aware Prompt Optimization for LLM Agents

Hard Rules, Soft Preferences: Bridging Reasoning, Learning, and Optimization for Personalized Packing Checklist Generation

CARL: Constraint-Aware Reinforcement Learning for Planning with LLMs