arXiv Computation and Language October 16, 2026 By Lei Xu, Xin Quan, Andr\'e Freitas

Reasoning without Gold Standards: A Proxy-Judge Theory of Autoformalization

Read the original on arXiv Computation and Language →

The paper introduces a reference‑free proxy‑judge framework for autoformalization, separating correctness into grounding, statement faithfulness, coverage, and inference. It presents three theoretical results linking proxy error rates to pass‑but‑wrong rates, the impact of false rejections on refinement, and a decomposition of final pass rates. The framework is instantiated with eight proxies and evaluated on ProofNet, miniF2F, e‑SNLI, and ProntoQA, showing that the predictions hold and identifying failure routes.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI

4d ago

claimcheck: Measuring and Improving Specification Faithfulness

arXiv:2610.17358v1 Announce Type: cross Abstract: Formal verification proves that a program satisfies its specification, but says nothing about whether the specification captures what the author mean...

By Martin Amster, One An, Holly Baker, Fernanda Graciolli, Simon Henniger, Nada Amin

llms benchmarks

arXiv Computation and Language

Sep 16

Autoformalizing Argumentative Material Inferences

The paper introduces GUARD, a neuro‑symbolic system that autoformalizes argumentative material by completing missing premises (guards) before formal verification. It uses large language models to generate candidate guards, Isabelle/HOL to verify them, and a contrastive test to ensure the proof depends on the original premises and does not over‑generalize. Experiments on Debatepedia and ARCT show that GUARD improves verified‑faithful scores by over 30 points and reduces leakage by about 20 points compared to prior LLM‑driven theorem proving methods.

By Xin Quan, Reto Gubelmann, Andr\'e Freitas

llms benchmarks

arXiv AI

Jul 16

Interventional Grounding Audits: Black-Box Premise-Dependency Tests for LLM Chain-of-Thought via Predicate Substitution

arXiv:2607. 13069v1 Announce Type: new Abstract: Large language models produce chain-of-thought (CoT) reasoning that appears logically sound yet may not genuinely depend on its stated premises.

By Hironao Nakamura

llms benchmarks

arXiv AI

Jul 15

Evidence-Grounded Verified Agentic Reasoning: A Path Toward Eliminating LLM Hallucination in Empirical Inference via Tool-Attested Kernel Proofs

arXiv:2607. 12650v1 Announce Type: cross Abstract: Tool access alone does not make LLM empirical reasoning governable: accepted outputs need not descend from attested evidence, and accepted deductions need not hold up under formal scrutiny.

By Junyu Ren

llms agents safety

arXiv AI

Aug 18

Does the Proof Prove It That Way? Faithful Formalization of Elements Proofs

arXiv:2608. 15432v1 Announce Type: new Abstract: In formal verification, both the autoformalization of statements and automated proof search have been studied extensively.

By Tadd Mao, Tianjun Zhong, Dhruva Arekar, Yuming Feng, One An, Jiani Huang, Xujie Si, Ziyang Li

llms agents

arXiv AI

Sep 21

SWE-Proof: Can Language Models Resolve Real-World Issues with Machine-Checked Proofs?

arXiv:2609.21190v1 Announce Type: cross Abstract: Ensuring the correctness of LLM-generated code is a core challenge for modern software engineering. Benchmarks for agentic code generation check corr...

By George Ma, Benjamin Mikek, Haoyu Li, Ferhat Erata, Yuhao Zhang, Zeren Shui, Behrooz Omidvar Tehrani, Jun Huan, Murali Krishna Ramanathan, Somayeh Sojoudi, Hao Zhou, Anoop Deoras

llms agents benchmarks safety

One email a morning, machine-written

Related stories

claimcheck: Measuring and Improving Specification Faithfulness

Autoformalizing Argumentative Material Inferences

Interventional Grounding Audits: Black-Box Premise-Dependency Tests for LLM Chain-of-Thought via Predicate Substitution

Evidence-Grounded Verified Agentic Reasoning: A Path Toward Eliminating LLM Hallucination in Empirical Inference via Tool-Attested Kernel Proofs

Does the Proof Prove It That Way? Faithful Formalization of Elements Proofs

SWE-Proof: Can Language Models Resolve Real-World Issues with Machine-Checked Proofs?