arXiv AI
5d ago
Are you Talking Logic to Me? Assessing Language Models Syllogistic Reasoning Capabilities
arXiv:2608. 12374v1 Announce Type: cross Abstract: Language models (LMs) struggle with logical tasks like reasoning on syllogisms.
SEF-CLGC at SemEval-2026 Task 11: Logical Notation Impact on Language Model Performance
arXiv:2606. 09157v1 Announce Type: cross Abstract: This paper revisits our pipeline called Syllogistic Evaluation Framework-Common Logic Grammar Construction (SEF-CLGC).
Related stories
arXiv AI
Jul 17
HABIB_TAZ at SemEval-2026 Task 11: Disentangling Formal Logic from Content via Synthetic Training and Multi-Objective Optimization
arXiv:2607. 14349v1 Announce Type: cross Abstract: While Large Language Models (LLMs) excel in many general NLP tasks, their formal reasoning capabilities are often compromised by content effects, demonstrating a measurable bias towards real-world plausibility.
arXiv AI
Jun 19
QMFOL: Benchmarking Large Language Model Reasoning via Quantifiable Monadic First-Order Logic Test Case Generation
arXiv:2606. 20227v1 Announce Type: new Abstract: Large Language Models (LLMs) have made significant progress in reasoning, particularly in deductive reasoning, which is crucial for high-stakes decision-making.
arXiv AI
Jul 28
Do Diagrams Help Large Language Models Reason? Evidence from Syllogistic Reasoning
arXiv:2607. 23513v1 Announce Type: cross Abstract: Diagrams are widely used to support logical reasoning, and prior studies suggest that representations such as Euler diagrams can improve human reasoning performance.
arXiv Machine Learning
Jun 16
Pushing the Boundaries of Natural Reasoning: Interleaved Bonus from Formal-Logic Verification
arXiv:2601. 22642v2 Announce Type: replace Abstract: Large Language Models (LLMs) show remarkable capabilities, yet their stochastic next-token prediction creates logical inconsistencies and reward hacking that formal symbolic systems avoid.
Hugging Face Trending Papers
Jul 26
Do Diagrams Help Large Language Models Reason? Evidence from Syllogistic Reasoning
Diagrams are widely used to support logical reasoning, and prior studies suggest that representations such as Euler diagrams can improve human reasoning performance. Recent work has also explored their effects on large language models (LLMs).
arXiv Machine Learning
Jul 21
LogicIF: Towards Complex Logic Instruction Following
arXiv:2508. 09125v3 Announce Type: replace-cross Abstract: Instruction following has catalyzed the recent era of Large Language Models (LLMs) and is the foundational skill underpinning more advanced capabilities such as reasoning and agentic behaviors.
arXiv AI
Jun 3
Reasoning Structure of Large Language Models
arXiv:2606. 03883v1 Announce Type: new Abstract: Large reasoning models (LRMs) are often evaluated using metrics such as final-answer accuracy or token count.
arXiv AI
Jul 8
PluraMath: Extending Mathematical Reasoning Evaluation Beyond High-Resource Languages
arXiv:2607. 05992v1 Announce Type: cross Abstract: Mathematical reasoning has become a central task for evaluating and tuning reasoning Large Language Models (LLMs), yet existing benchmarks remain heavily biased toward high-resource languages, with English and Chinese dominating both pre-training corpora and evaluation suites.
arXiv AI
Jul 17
Step-Tagging: Toward controlling the generation of Language Reasoning Models through step monitoring
arXiv:2512. 14332v2 Announce Type: replace-cross Abstract: The field of Language Reasoning Models (LRMs) has been very active over the past few years with advances in training and inference techniques enabling LRMs to reason longer, and more accurately.
arXiv AI
Jul 31
Making Implicit Premises Explicit in Logical Understanding of Enthymemes
arXiv:2603. 06114v2 Announce Type: replace-cross Abstract: Real-world arguments in text and dialogues are normally enthymemes (i.
arXiv AI
Jun 3
Code-on-Graph: Iterative Programmatic Reasoning via Large Language Models on Knowledge Graphs
arXiv:2606. 03705v1 Announce Type: new Abstract: Knowledge Graphs (KGs) are widely used to mitigate the limitations of Large Language Models (LLMs), such as outdated knowledge and hallucinations.