Competence-Based Evaluation of Logical Reasoning in Large Language Models

The paper proposes a competence‑based evaluation (CBE) framework for assessing logical reasoning in large language models. CBE tests whether models consistently apply the same reasoning competence across a family of problems, using a parameterized generator to vary complexity while preserving the underlying rule. Experiments on GPT and Qwen models show that even strong models often fail to meet the competence condition across the tested family, indicating limitations in generalization.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• fine-tuning
• benchmarks