arXiv AI
What Was Said, Not What Was 'Thought': Type-6 Logic for CoT Verification
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Aug 5
The Tell-Tale Trace: Detecting Reasoning Failures in LLMs Using Chain-of-Thought Dynamics
arXiv:2608. 03291v1 Announce Type: cross Abstract: Chain-of-thought (CoT) reasoning improves large language model (LLM) performance while also providing an observable interface to the model's reasoning process.
arXiv AI
Jul 15
Evidence-Grounded Verified Agentic Reasoning: A Path Toward Eliminating LLM Hallucination in Empirical Inference via Tool-Attested Kernel Proofs
arXiv:2607. 12650v1 Announce Type: cross Abstract: Tool access alone does not make LLM empirical reasoning governable: accepted outputs need not descend from attested evidence, and accepted deductions need not hold up under formal scrutiny.
arXiv AI
Sep 24
Not What You Meant: Can LLMs Follow a Specified Negation Semantics?
The paper investigates how large language models (LLMs) interpret negation across different logical semantics—open‑world vs. closed‑world, two‑ vs. three‑valued, and credulous vs. skeptical reasoning. Using the newly introduced NAFBench, a procedural generator that creates solver‑certified logic programs and their natural‑language verbalizations, the authors evaluate LLMs on four semantic viewpoints (SLDNF, well‑founded semantics, and stable‑model semantics). Results show a persistent gap: even the strongest models achieve only 59–74% accuracy, with many models sensitive to rule ordering and prone to overcommitment on undefined cases, though some frontier models reach near‑perfect performance on a fixed‑complexity set. "whyItMatters":"The study highlights that current LLMs struggle to reliably follow explicitly specified negation semantics, underscoring a limitation in their logical reasoning capabilities."
arXiv AI
Jul 14
Think Through a Bottleneck: Hourglass Reasoning for Rigorous Induction
arXiv:2607. 11696v1 Announce Type: new Abstract: Self-refinement often fails to strengthen few-shot inductive reasoning in large language models.
arXiv Computation and Language
Sep 16
Autoformalizing Argumentative Material Inferences
The paper introduces GUARD, a neuro‑symbolic system that autoformalizes argumentative material by completing missing premises (guards) before formal verification. It uses large language models to generate candidate guards, Isabelle/HOL to verify them, and a contrastive test to ensure the proof depends on the original premises and does not over‑generalize. Experiments on Debatepedia and ARCT show that GUARD improves verified‑faithful scores by over 30 points and reduces leakage by about 20 points compared to prior LLM‑driven theorem proving methods.
arXiv Computation and Language
Aug 21
Grading the Graders: Verification Autonomy Levels (L0-L5) for LLM Reasoning
arXiv:2608. 19009v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly paired with verifiers (step checkers, self-consistency filters, tool-based fact checkers, formal proof assistants) that claim to detect the model's errors.