arXiv Machine Learning October 5, 2026 By Jinwoo Kim

Constraint-Aware Training

Read the original on arXiv Machine Learning →

The paper introduces a constraint‑aware training objective that externalizes program analysis constraints used during inference, avoiding redundant learning of these constraints during model training. It presents three theorems demonstrating that this approach can reduce model size and improve data efficiency. A controlled synthetic experiment confirms that constraint‑aware training achieves lower prediction loss than standard cross‑entropy training at the same parameter count and data volume.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

llms

arXiv AI

Jun 30

The Hidden Cost of Structured Generation in LLMs: Draft-Conditioned Constrained Decoding

arXiv:2603. 03305v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used to generate executable outputs, JSON objects, and API calls, where a single syntax error can make the output unusable.

By Avinash Reddy, Thayne T. Walker, James S. Ide, Amrit Singh Bedi

llms benchmarks

arXiv AI

Sep 4

RECAST: Expanding the Boundaries of LLMs' Complex Instruction Following with Multi-Constraint Data

RECAST is a new framework that generates datasets with far more constraints per example than existing benchmarks, aiming to push large language models (LLMs) to better follow complex instructions. The authors built RECAST-30K, a 30,000‑instance dataset covering 19 constraint types extracted from real prompt‑response pairs, and showed that fine‑tuning on it improves LLMs’ ability to handle complex tasks without harming general performance. RECAST also provides rule‑based and LLM‑based validators for automatic constraint verification, enabling reward‑based reinforcement learning to further enhance model performance on challenging tasks.

By Zhengkang Guo, Wenhao Liu, Mingchen Xie, Jingwen Xu, Zisu Huang, Muzhao Tian, Jianhan Xu, Yuanzhe Shen, Qi Qian, Muling Wu, Xiaohua Wang, Changze Lv, He-Da Wang, Hu Yao, Xiaoqing Zheng, Xuanjing Huang

llms reinforcement-learning benchmarks

arXiv AI

Jun 4

Beyond Objective Equivalence: Constraint Injection for LLM-Based Optimization Modeling on Vehicle Routing Problems

arXiv:2606. 04816v1 Announce Type: new Abstract: Large language models (LLMs) increasingly translate natural-language optimization problems into executable solver code.

By Xizi Luo, Changhong He, Dongdong Geng, Chenggong Shi, Yu Mei

llms benchmarks

arXiv Machine Learning

Sep 23

Towards Mitigating Excessive Forgetting in LLM Unlearning via Entanglement-Guidance with Proxy Constraint

arXiv:2508.20443v3 Announce Type: replace Abstract: Large language models (LLMs) are trained on massive datasets that may include private or copyrighted content. Due to growing privacy and ownership...

By Zhihao Liu, Jian Lou, Yuke Hu, Xiaochen Li, Yitian Chen, Tailun Chen, Zhizhen Qin, Kui Ren, Zhan Qin

llms benchmarks safety

arXiv AI

Oct 2

Auto-Formalizing Neuro-Symbolic Predictors

arXiv:2610.01519v1 Announce Type: cross Abstract: Neuro-Symbolic (NeSy) predictors incorporate prior knowledge into the prediction process of neural networks, ensuring that outputs satisfy specified...

By Samuele Bortolotti, Weixin Chen, Han Zhao, Andrea Passerini, Stefano Teso, Antonio Vergari

llms benchmarks

arXiv AI

Aug 18

CAPO: Constraint-Aware Prompt Optimization for LLM Agents

arXiv:2608. 16068v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed as agents that rely on system prompts to use tools and complete tasks.

By Victor Ye Dong, Reid Pryzant, Yi Liu, Jian Jiao

llms agents benchmarks safety

One email a morning, machine-written

Related stories

The Hidden Cost of Structured Generation in LLMs: Draft-Conditioned Constrained Decoding

RECAST: Expanding the Boundaries of LLMs' Complex Instruction Following with Multi-Constraint Data

Beyond Objective Equivalence: Constraint Injection for LLM-Based Optimization Modeling on Vehicle Routing Problems

Towards Mitigating Excessive Forgetting in LLM Unlearning via Entanglement-Guidance with Proxy Constraint

Auto-Formalizing Neuro-Symbolic Predictors

CAPO: Constraint-Aware Prompt Optimization for LLM Agents