arXiv Computation and Language October 9, 2026 By Andrea Brunello, Cristian Curaba, Luca Geatti, Michele Mignani, Angelo Montanari, Nicola Saccomanno

Natural Language to First-Order Logic LLM-based Autoformalization

Read the original on arXiv Computation and Language →

The paper introduces a clear definition for the task of converting natural language into First‑Order Logic (FOL), separating ontology extraction from logical translation to improve evaluation consistency. It surveys current datasets, evaluation metrics, and LLM‑based approaches—including fine‑tuning, prompting, and verification‑based refinement. The authors also outline remaining challenges in benchmarking, semantic evaluation, ontology‑aware methods, and practical applications.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI

Sep 1

Beyond Surface Forms: Symbolic Edits as a Test for Logical Reasoning with LLMs

arXiv:2608.30256v1 Announce Type: cross Abstract: Logical reasoning with large language models (LLMs) is a critical capability, as it reflects a system's ability to correctly deduce hypotheses from a...

By Ramya Keerthy Thatikonda, Wray Buntine, Ehsan Shareghi

llms

arXiv Computation and Language

Oct 5

Last Layer Logits to Logic: Empowering LLMs with Logic-Consistent Structured Knowledge Reasoning

arXiv:2511.07910v3 Announce Type: replace Abstract: Large Language Models (LLMs) achieve excellent performance in natural language reasoning tasks through pre-training on vast unstructured text, enab...

By Songze Li, Zhiqiang Liu, Zhaoyan Gong, Xiaoke Guo, Zhongpu Bo, Zhengke Gui, Lei Liang, Huajun Chen, Wen Zhang

llms nlp benchmarks

arXiv AI

Jun 19

QMFOL: Benchmarking Large Language Model Reasoning via Quantifiable Monadic First-Order Logic Test Case Generation

arXiv:2606. 20227v1 Announce Type: new Abstract: Large Language Models (LLMs) have made significant progress in reasoning, particularly in deductive reasoning, which is crucial for high-stakes decision-making.

By Xinyi Zheng, Ling Shi, Tianlong Yu, Yongxin Zhao, Lorenz Goette, Kailong Wang

llms benchmarks

arXiv Machine Learning

Jul 21

LogicIF: Towards Complex Logic Instruction Following

arXiv:2508. 09125v3 Announce Type: replace-cross Abstract: Instruction following has catalyzed the recent era of Large Language Models (LLMs) and is the foundational skill underpinning more advanced capabilities such as reasoning and agentic behaviors.

By Mian Zhang, Shujian Liu, Sixun Dong, Ming Yin, Yebowen Hu, Xun Wang, Simin Ma, Song Wang, Sathish Reddy Indurthi, Haoyun Deng, Zhiyu Zoey Chen, Kaiqiang Song

llms agents benchmarks

arXiv AI

Oct 23

When Form Changes but Logic Doesn't: Building Logic-invariant LLMs through Logical Structure

The paper investigates whether large language models (LLMs) truly grasp logical structure or merely rely on surface semantics. By evaluating models on logically equivalent inputs, the authors find that many LLMs fail to produce consistent predictions and struggle with unseen logical forms. They introduce LoGIST, a framework that maps inputs to decision diagrams and uses these logical embeddings to guide LLM reasoning, thereby reducing inconsistency and improving generalization.

By Xuyuan Liu, Xinshuai Dong, Elynn Chen, Yujun Yan

llms rag

arXiv AI

Jul 23

Logic-Guided Data Extraction with Answer Set Programming and Large Language Models

arXiv:2607. 19365v1 Announce Type: new Abstract: When Large Language Models (LLMs) are used for semantic data extraction from unstructured text, producing candidate relational facts from natural language, they may remain unreliable for tasks requiring complex combinatorial reasoning and global consistency.

By Mario Alviano, Lorenzo Grillo, Nicola Leone, Fabrizio Lo Scudo

llms benchmarks

One email a morning, machine-written

Related stories

Beyond Surface Forms: Symbolic Edits as a Test for Logical Reasoning with LLMs

Last Layer Logits to Logic: Empowering LLMs with Logic-Consistent Structured Knowledge Reasoning

QMFOL: Benchmarking Large Language Model Reasoning via Quantifiable Monadic First-Order Logic Test Case Generation

LogicIF: Towards Complex Logic Instruction Following

When Form Changes but Logic Doesn't: Building Logic-invariant LLMs through Logical Structure

Logic-Guided Data Extraction with Answer Set Programming and Large Language Models