BANGLAPLAN is a new benchmark comprising 1,000 Bangla planning and decision‑making problems across ten everyday task families, presented in four language registers (everyday, formal, colloquial, and Bangla–English mixed). The benchmark includes both feasible and infeasible instances and requires answers in eight structured JSON formats such as schedules, routes, and allocations. Evaluation of 11 language‑model configurations shows a wide performance gap, with GPT‑6.1 Sol achieving 98.7% accuracy while most models perform poorly, highlighting the challenge of constraint‑based planning in Bangla.
arXiv:2601. 09097v3 Announce Type: replace Abstract: Multi-constraint planning involves identifying, evaluating, and refining candidate plans while satisfying multiple, potentially conflicting constraints.
The evaluation of mathematical reasoning in large language models (LLMs) has predominantly focused on high-resource languages like English. This has created a significant barrier to the equitable development and deployment of AI in linguistically diverse regions such as Bangladesh, where over 230 million people speak Bengali.
arXiv:2508.01300v2 Announce Type: replace Abstract: AI planning provides agents the ability to act autonomously towards achieving a goal in a given environment. Recent advances in Large Language Mode...
arXiv:2610.08246v1 Announce Type: new Abstract: Frontier large language models (LLMs) can generate heuristic functions that guide search to achieve state-of-the-art performance in satisficing plannin...
CCTU is a new benchmark designed to evaluate large language models (LLMs) on their ability to use tools under complex constraints. It includes 200 test cases that average seven constraint types and 4,700‑token prompts, covering resource, behavior, toolset, and response dimensions. An executable validation module performs step‑level checks, and nine state‑of‑the‑art LLMs were tested, revealing that none exceed a 20% task completion rate when strict constraints are enforced, with frequent violations and limited self‑refinement.
arXiv:2606. 01046v1 Announce Type: new Abstract: The development of Large Language Models (LLMs) has significantly improved travel planning applications, yet evaluating such models is limited by existing benchmarks' limitations: 1) overemphasis on constraint compliance, neglecting multi-dimensional qualities like spatio-temporal cost; 2) datasets lacking real-world authenticity and coverage in key areas (e.
LangNCO is a framework that transforms natural-language constraints into structured inference-time interventions for a frozen neural combinatorial optimization (NCO) solver. It uses a pretrained large language model to convert each requirement into a domain-specific language rule set, which a deterministic Rule Executor applies as hard action masks. Critical-Offset Decoding then reallocates rollout budget to ensure feasibility, achieving near-perfect feasibility across multiple constraint types without retraining the solver.
arXiv:2607. 22621v1 Announce Type: new Abstract: While large language models (LLMs) enable strong question answering (QA), budgeted deployment is complicated by nondeterminism and heterogeneous resource profiles (cost, latency, and energy).
The paper introduces SDDL, a neuro‑symbolic framework that converts natural‑language combinatorial scheduling problems into compact, solver‑aligned representations, delegating low‑level modeling and search to a deterministic compiler and external solver. On a 300‑instance subset of scheduling tasks, SDDL achieves higher feasibility rates for resource‑constrained language models—up to 55.3% and 28.3%—compared to direct‑generation baselines (23.7% and 1.3%) and solver‑code baselines (21.7% and 7.0%), with a median optimality gap of 0.0% among feasible schedules.
SCHEDBench is a natural‑language benchmark that evaluates whether large language models (LLMs) produce schedules that remain constraint‑feasible when the same scheduling problem is expressed in different natural‑language surface forms. The benchmark covers 1,132 instances from job‑shop scheduling, resource‑constrained project scheduling, nurse rostering, and curriculum timetabling, and uses domain‑specific templates and surface‑form variations to generate varied problem statements. Experiments with thirteen frontier and open‑weight LLMs show that models are not reliably invariant to semantically equivalent renderings, with surface‑form variation reducing feasibility and increasing hard‑constraint violations, especially when constraints are reordered.
GRASP is a multi-stage planning framework that improves the reliability of large language models on complex tasks. It separates planning into three specialized modules—GenPlan for global macro-guidelines, RevPlan for exploring localized strategies, and VerPlan for multi-criteria evaluation—allowing context isolation and strict macro-regularization. Experiments show GRASP outperforms direct LLM planners by significant margins on datasets such as Natural Plan Calendar Scheduling, ZebraLogic, and SciBench Math, and it mitigates performance collapse in multi-task and dual-task settings.
arXiv:2606. 05792v1 Announce Type: cross Abstract: TLA+ has supported industrial verification at companies such as Amazon and Microsoft, yet writing correct TLA+ specifications from natural language still requires time and expertise, which limits adoption.