The paper introduces Constraint‑Guided Enterprise Data Mapping (CGM), a neuro‑symbolic approach that uses schema‑grounded admissibility constraints to steer large language models (LLMs) in aligning enterprise data. CGM operates in three stages: defining constraints with metadata, generating candidates under relaxed constraints to ensure feasibility, and ranking them with a bounded LLM. Experiments show that hard constraints dramatically reduce candidate space and improve F1 scores, enabling small models to match or surpass large LLMs at a fraction of the cost while reducing expert effort.
By Sebastian Monka, Pramod Anantharam, Thien Vo Minh, Lavdim Halilaj
CCTU is a new benchmark designed to evaluate large language models (LLMs) on their ability to use tools under complex constraints. It includes 200 test cases that average seven constraint types and 4,700‑token prompts, covering resource, behavior, toolset, and response dimensions. An executable validation module performs step‑level checks, and nine state‑of‑the‑art LLMs were tested, revealing that none exceed a 20% task completion rate when strict constraints are enforced, with frequent violations and limited self‑refinement.
By Junjie Ye, Guoqiang Zhang, Wenjie Fu, Zelin Li, Tao Gui, Qi Zhang, Xuanjing Huang
arXiv:2607. 26220v1 Announce Type: cross Abstract: Context: Large Language Models (LLMs) offer natural-language flexibility for automated requirements elicitation but frequently generate structurally invalid requirements and logical inconsistencies, lacking formal correctness guarantees.
By Ahmed Ibrahim
arXiv:2607. 23386v1 Announce Type: new Abstract: We document a failure class in frontier large language models -- exception chain collapse -- observed in eligibility evaluation under nested conditional rules of the form "A is required UNLESS B applies, UNLESS C overrides B".
By Paul Simpson, John Kozak, Lisa Doake
arXiv:2608. 04719v1 Announce Type: new Abstract: Agent evaluations tell us that a model picked the wrong tool, but rarely why.
By Atul Anand, Sourav Chattaraj
Agent evaluations tell us that a model picked the wrong tool, but rarely why. We introduce canary tools: diagnostic probe tools planted in an agent's Model Context Protocol (MCP) tool set, each engineered to probe one specific tool-selection weakness.