The paper introduces a clear definition for the task of converting natural language into First‑Order Logic (FOL), separating ontology extraction from logical translation to improve evaluation consistency. It surveys current datasets, evaluation metrics, and LLM‑based approaches—including fine‑tuning, prompting, and verification‑based refinement. The authors also outline remaining challenges in benchmarking, semantic evaluation, ontology‑aware methods, and practical applications.
arXiv:2608.30256v1 Announce Type: cross Abstract: Logical reasoning with large language models (LLMs) is a critical capability, as it reflects a system's ability to correctly deduce hypotheses from a...
arXiv:2511.07910v3 Announce Type: replace Abstract: Large Language Models (LLMs) achieve excellent performance in natural language reasoning tasks through pre-training on vast unstructured text, enab...
arXiv:2606. 20227v1 Announce Type: new Abstract: Large Language Models (LLMs) have made significant progress in reasoning, particularly in deductive reasoning, which is crucial for high-stakes decision-making.
arXiv:2508. 09125v3 Announce Type: replace-cross Abstract: Instruction following has catalyzed the recent era of Large Language Models (LLMs) and is the foundational skill underpinning more advanced capabilities such as reasoning and agentic behaviors.
The paper investigates whether large language models (LLMs) truly grasp logical structure or merely rely on surface semantics. By evaluating models on logically equivalent inputs, the authors find that many LLMs fail to produce consistent predictions and struggle with unseen logical forms. They introduce LoGIST, a framework that maps inputs to decision diagrams and uses these logical embeddings to guide LLM reasoning, thereby reducing inconsistency and improving generalization.
arXiv:2607. 19365v1 Announce Type: new Abstract: When Large Language Models (LLMs) are used for semantic data extraction from unstructured text, producing candidate relational facts from natural language, they may remain unreliable for tasks requiring complex combinatorial reasoning and global consistency.
arXiv:2606. 30441v1 Announce Type: cross Abstract: A rigorous formalization of system requirements is a fundamental prerequisite for the verification of Multi-Agent Systems (MAS).
The paper introduces an epistemically and formally grounded ensemble (EFG) of large language model judges to evaluate autoformalization tasks in formal mathematics. It defines four criteria—logical preservation, mathematical consistency, formal quality, and formal validity—to provide a transparent, multi‑granular assessment. Experiments show that this ensemble outperforms coarse‑grained models, offering a scalable and interpretable proxy for evaluating formal mathematical reasoning.
The paper investigates when intermediate reasoning traces benefit machine translation by examining models, languages, domains, and datasets. It finds that the optimal reasoning language depends on the model, reasoning length has a non‑monotonic effect on quality, and traces display recurring functional patterns. Using Hierarchical Meta‑Summarization, the authors uncover a shared structure of understanding/planning, translating/drafting, and refining/verifying, while also noting domain‑specific variations, suggesting that reasoning should be tailored rather than uniformly applied.
arXiv:2605. 23965v2 Announce Type: replace Abstract: Large Language Models (LLMs) achieve strong performance on logical reasoning benchmarks, yet their reliability remains uncertain.
The paper introduces GLaMoR, a reasoning pipeline that converts OWL ontologies into graph-structured data and applies a Graph Language Model (GLM) for consistency checking. It addresses the challenge of verifying ontology consistency, especially for large ontologies where classical reasoners become computationally expensive. Experiments on NCBO BioPortal ontologies show that GLaMoR achieves 95% accuracy and is 20 times faster than traditional reasoners.
arXiv:2608.22974v1 Announce Type: new Abstract: Large language model (LLM) agents rely heavily on knowledge encoded in model parameters or presented as unstructured context. In domain-specific tasks,...