arXiv AI
Beyond Surface Forms: Symbolic Edits as a Test for Logical Reasoning with LLMs
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 19
QMFOL: Benchmarking Large Language Model Reasoning via Quantifiable Monadic First-Order Logic Test Case Generation
arXiv:2606. 20227v1 Announce Type: new Abstract: Large Language Models (LLMs) have made significant progress in reasoning, particularly in deductive reasoning, which is crucial for high-stakes decision-making.
arXiv AI
Oct 23
When Form Changes but Logic Doesn't: Building Logic-invariant LLMs through Logical Structure
The paper investigates whether large language models (LLMs) truly grasp logical structure or merely rely on surface semantics. By evaluating models on logically equivalent inputs, the authors find that many LLMs fail to produce consistent predictions and struggle with unseen logical forms. They introduce LoGIST, a framework that maps inputs to decision diagrams and uses these logical embeddings to guide LLM reasoning, thereby reducing inconsistency and improving generalization.
arXiv AI
Jun 3
X-RAY: Mapping LLM Reasoning Capability via Formalized and Calibrated Probes
arXiv:2603. 05290v2 Announce Type: replace Abstract: Large language models (LLMs) achieve promising performance, yet their ability to reason remains poorly understood.
arXiv AI
Jun 12
LLMs as ASP Programmers: Self-Correction Enables Task-Agnostic Nonmonotonic Reasoning
arXiv:2604. 27960v2 Announce Type: replace Abstract: Recent large language models (LLMs) have achieved impressive reasoning milestones but continue to struggle with high computational costs, logical inconsistencies, and sharp performance degradation on high-complexity problems.
arXiv AI
Jul 8
Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges
arXiv:2605. 19723v2 Announce Type: replace-cross Abstract: Mathematical reasoning is essential for problem-solving in education, science, and industry, serving as a crucial benchmark for evaluating artificial intelligence systems.
arXiv Computation and Language
Aug 28
Neuro-symbolic PRM: Enhancing Scientific Reasoning via Structured Traces and Symbolic Verification
The paper introduces a neuro‑symbolic framework for scientific reasoning that separates symbolic validity and semantic groundedness. A deterministic symbolic verifier acts as a hard filter to guarantee syntactic and arithmetic correctness, while a Process Reward Model (PRM) is trained on verifier‑accepted steps to assess contextual grounding. The authors propose Counterfactual Symbolic Perturbation (CSP) to generate hard negative examples that pass the verifier but are logically flawed, enabling efficient PRM training and a verifier‑first constrained search at inference.