arXiv AI By Noor Islam S. Mohammad, Tamim Sheikh

The Faithfulness Gap: Certifying Semantic Equivalence Between Natural-Language and Formal Mathematical Statements

Read the original on arXiv AI →

arXiv:2606. 16541v1 Announce Type: new Abstract: Autoformalization, translating natural-language mathematics into formal proof assistants, is bottlenecked not by translation fluency but by \emph{faithfulness}: a formal statement can typecheck and be provable, yet still encode a different theorem than the source intended.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language
Sep 16

Autoformalizing Argumentative Material Inferences

The paper introduces GUARD, a neuro‑symbolic system that autoformalizes argumentative material by completing missing premises (guards) before formal verification. It uses large language models to generate candidate guards, Isabelle/HOL to verify them, and a contrastive test to ensure the proof depends on the original premises and does not over‑generalize. Experiments on Debatepedia and ARCT show that GUARD improves verified‑faithful scores by over 30 points and reduces leakage by about 20 points compared to prior LLM‑driven theorem proving methods.

By Xin Quan, Reto Gubelmann, Andr\'e Freitas
arXiv AI
Jun 2

Formally Solving Answer-Construction Problems in Lean

arXiv:2505. 18492v5 Announce Type: replace Abstract: Mathematical competition problems fall into two broad types: theorem proving, which asks for a proof of a given statement, and answer construction, which requires constructing a property-satifying object with proofs.

By Jialiang Sun, Yuzhi Tang, Ao Li, Chris J. Maddison, Kuldeep S. Meel
arXiv Computation and Language
Sep 22

Euston: Training Away Mathematical Sycophancy Without Losing the Mathematics

Euston is an 8‑B parameter mathematical claim‑verification model that resists producing false derivations when presented with corrupted theorems. It was trained on 3,026 matched true/corrupted statement pairs generated by GraphSynth, a probabilistic factor‑graph generator, and fine‑tuned from DeepSeek‑R1‑8B using GRPO. On a balanced held‑out split, Euston’s balanced accuracy rose from 29.50 % to 63.75 %, and its discrimination gap improved from –0.5 % to +27.5 %, while maintaining comparable general mathematical ability and reducing response length and truncation rates.

By Zehua Cheng, Wei Dai, Jiahao Sun
arXiv Machine Learning
Sep 1

Where Induction Runs Out: Description-Length Difficulty and the Memorisation Gap in Integer-Sequence Benchmarks

The paper investigates integer‑sequence benchmarks from the OEIS by applying a two‑part minimum description length (MDL) learner that searches for P‑recursive recurrences. It finds that MDL difficulty correlates with a combinatorial parameter count, that most sequences fit a recurrence on a prefix but not at full length (the “wilderness” regime), and that language models do not hallucinate in the wilderness but instead hedge, showing that memorisation dominates perceived competence. The study provides a cheap, contamination‑free difficulty signal for OEIS‑derived benchmarks.

By Sabilashan Ganeshan
arXiv AI
Aug 17

MathForm: Scaling Mathematical Autoformalization with Knowledge Retrieval and Verification-Guided Refinement

arXiv:2608. 14221v1 Announce Type: new Abstract: Autoformalization is commonly framed as translating natural-language mathematical statements into machine-verifiable formal languages such as Lean 4.

By Lushi Pu, Weiming Zhang, Xinheng Xie, Zixuan Fu, Bingxiang He, Hengyu Zhao, Hongya Lyu, Xin Li, Jie Zhou, Yudong Wang