arXiv AI July 21, 2026 By Haifeng Li, Mo Hai

Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits

arXiv:2607. 16646v1 Announce Type: cross Abstract: Large language models now translate natural-language descriptions of decision problems into solver-ready optimization models, but they fail silently.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Sep 1

Beyond the Answer Key: Robustness Evaluation of Large Language Models for Step-Level Mathematical Verification

arXiv:2608.28725v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as graders, verifiers, and process auditors, but most mathematical evaluations still emphasize final...

By Fateme Mazdarani, Carlos Toxtli

llms fine-tuning efficiency benchmarks

arXiv AI

Aug 18

ReLoop: Structured Modeling and Behavioral Verification for Reliable LLM-Based Optimization

arXiv:2602. 15983v3 Announce Type: replace-cross Abstract: Large language models (LLMs) can translate natural language into optimization code, but silent failures pose a critical risk: code that executes and returns solver-feasible solutions may encode semantically incorrect formulations---a feasibility--correctness gap reaching 90 percentage points on compositional problems.

By Junbo Jacob Lian, Yujun Sun, Huiling Chen, Chaoyu Zhang, Hanzhang Qin, Chung-Piaw Teo

llms fine-tuning benchmarks

arXiv Machine Learning

Jun 5

TLA-Prover: Verifiable TLA+ Specification Synthesis via Preference-Optimized Low-Rank Adaptation

arXiv:2606. 06133v1 Announce Type: cross Abstract: TLA+ is a formal specification language for verifying distributed systems and safety-critical protocols.

By Eric Spencer, Arslan Bisharat, Brian Ortiz, Khushboo Bhadauria, TaiNing Wang, George K. Thiruvathukal, Konstantin Laufer, Mohammed Abuhamad

llms reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Sep 18

Large Language Models as Falsifiers for Cyber-Physical Systems

The paper introduces LLM-Falsifier, a large language model–based method for falsifying cyber‑physical system specifications written in Signal Temporal Logic (STL). By exposing the LLM to semantic cues such as natural‑language names, output trajectories, and critical‑time witnesses, the approach performs smarter, sample‑efficient robustness searches. On ARCH‑COMP benchmarks, LLM‑Falsifier outperforms existing tools across 14 of 21 specifications, requiring fewer simulations to find counterexamples.

By Ali ArjomandBigdeli, Jiawei Zhou, Stanley Bak

llms benchmarks

arXiv AI

Sep 21

SWE-Proof: Can Language Models Resolve Real-World Issues with Machine-Checked Proofs?

arXiv:2609.21190v1 Announce Type: cross Abstract: Ensuring the correctness of LLM-generated code is a core challenge for modern software engineering. Benchmarks for agentic code generation check corr...

By George Ma, Benjamin Mikek, Haoyu Li, Ferhat Erata, Yuhao Zhang, Zeren Shui, Behrooz Omidvar Tehrani, Jun Huan, Murali Krishna Ramanathan, Somayeh Sojoudi, Hao Zhou, Anoop Deoras

llms agents benchmarks safety

arXiv AI

4d ago

Beyond Rule-Based Mutation Testing: Test-Aware Mutant Generation Using Large Language Models

arXiv:2609.35841v1 Announce Type: cross Abstract: Mutation testing evaluates test-suite adequacy by injecting synthetic faults into program code. However, traditional rule-based tools often generate...

By Nils Kiele, Zainab Saad, Zirui Wang, Steve Drew, Samira Ebrahimi Kahou

llms benchmarks

One email a morning, machine-written

Related stories

Beyond the Answer Key: Robustness Evaluation of Large Language Models for Step-Level Mathematical Verification

ReLoop: Structured Modeling and Behavioral Verification for Reliable LLM-Based Optimization

TLA-Prover: Verifiable TLA+ Specification Synthesis via Preference-Optimized Low-Rank Adaptation

Large Language Models as Falsifiers for Cyber-Physical Systems

SWE-Proof: Can Language Models Resolve Real-World Issues with Machine-Checked Proofs?

Beyond Rule-Based Mutation Testing: Test-Aware Mutant Generation Using Large Language Models