arXiv:2607. 16646v1 Announce Type: cross Abstract: Large language models now translate natural-language descriptions of decision problems into solver-ready optimization models, but they fail silently.
arXiv:2608.28725v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as graders, verifiers, and process auditors, but most mathematical evaluations still emphasize final...
arXiv:2602. 15983v3 Announce Type: replace-cross Abstract: Large language models (LLMs) can translate natural language into optimization code, but silent failures pose a critical risk: code that executes and returns solver-feasible solutions may encode semantically incorrect formulations---a feasibility--correctness gap reaching 90 percentage points on compositional problems.
arXiv:2606. 06133v1 Announce Type: cross Abstract: TLA+ is a formal specification language for verifying distributed systems and safety-critical protocols.
The paper introduces LLM-Falsifier, a large language model–based method for falsifying cyber‑physical system specifications written in Signal Temporal Logic (STL). By exposing the LLM to semantic cues such as natural‑language names, output trajectories, and critical‑time witnesses, the approach performs smarter, sample‑efficient robustness searches. On ARCH‑COMP benchmarks, LLM‑Falsifier outperforms existing tools across 14 of 21 specifications, requiring fewer simulations to find counterexamples.
arXiv:2609.21190v1 Announce Type: cross Abstract: Ensuring the correctness of LLM-generated code is a core challenge for modern software engineering. Benchmarks for agentic code generation check corr...
arXiv:2609.35841v1 Announce Type: cross Abstract: Mutation testing evaluates test-suite adequacy by injecting synthetic faults into program code. However, traditional rule-based tools often generate...
arXiv:2608. 19009v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly paired with verifiers (step checkers, self-consistency filters, tool-based fact checkers, formal proof assistants) that claim to detect the model's errors.
The paper introduces LLM-Falsifier, a method that uses large language models to find counterexamples in cyber‑physical systems by minimizing Signal Temporal Logic robustness. By providing the LLM with natural‑language context, trajectory outputs, and critical‑time witnesses, the approach achieves smarter, more sample‑efficient searches. On ARCH‑COMP benchmarks, LLM‑Falsifier outperforms existing tools across 14 of 21 specifications in terms of simulations needed to locate a counterexample.
arXiv:2607. 29431v1 Announce Type: new Abstract: Large language models increasingly generate optimization models from natural language, but existing evaluation often reduces a generated model and its ground truth to a single equivalent/not-equivalent verdict or an execution-success rate--labels that are neither independently checkable nor faithful to the multiple distinct senses in which two formulations can agree.
arXiv:2606. 19588v1 Announce Type: new Abstract: Formal tools such as SAT and SMT solvers are increasingly embedded in language model reasoning pipelines when a safety or security critical question can be formulated in logic.
arXiv:2603. 25450v2 Announce Type: replace Abstract: Detecting when a language model is wrong without ground truth labels is a fundamental challenge for safe deployment.
arXiv:2606. 29441v1 Announce Type: cross Abstract: Inference-time safety methods for large language models have proliferated, yet no systematic comparison exists.