← Back to all news
arXiv AI By Yunxin Sun, Abulhair Saparov

Do Language Models Follow Occam's Razor? An Evaluation of Parsimony in Inductive and Abductive Reasoning

Read the original on arXiv AI →

The paper investigates whether large language models (LLMs) follow Occam's Razor when performing inductive and abductive reasoning. It introduces a synthetic framework for generating questions that require both types of reasoning and a new automated metric to evaluate the simplicity and correctness of generated hypotheses. Experiments show that while LLMs can handle simple scenarios, they struggle with complex world models and producing high‑quality, simplest hypotheses, even when using advanced reasoning techniques.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Oct 5

One Hypothesis Is Not Enough: Abductive Reasoning with Agentic Hypothesis Refinement over Knowledge Graphs

The paper introduces HypoAgent, an agentic framework for refining abductive hypotheses over knowledge graphs. It treats user-provided conditions as steering operators, using a small generator for initial proposals, root-cause analysis to identify problematic branches, and a refiner agent that revises hypotheses based on diagnosis and prior proposals. Experiments on BioKG, PharmKG8k, and DBpedia50 show HypoAgent surpasses one-shot generation in both single-turn and multi-turn scenarios, including unconditional settings.

By Yisen Gao, Yixi Cai, Tianshi Zheng, Jiaxin Bai, Yangqiu Song
llmsagents
More like this →
arXiv AI
Sep 7

Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Models

The paper surveys efficient reasoning in large language models, contrasting fast intuitive (System 1) and slow deep (System 2) reasoning. It analyzes why System 2 is computationally costly yet more accurate, and why System 1 is efficient but less effective. The survey covers causes of inefficiency, patterns of reasoning behavior, and potential solutions to balance performance and computational budgets, offering actionable insights and an open‑source repository for ongoing research.

By Rui Wang, Hongru Wang, Boyang Xue, Jianhui Pang, Shudong Liu, Yi Chen, Jiahao Qiu, Derek Fai Wong, Heng Ji, Kam-Fai Wong
llms
More like this →
arXiv Machine Learning
Oct 8

The Dichotomy Between Pattern Recognition and Step-by-Step Reasoning

The paper argues that pattern recognition and step‑by‑step reasoning lie on a spectrum, with large language models (LLMs) learning the latter when the next token depends on only a few preceding tokens. It formalises reasoning traces as paths on a De Bruijn graph, showing that the number of edges is far smaller than the number of possible traces, making step‑by‑step reasoning sample‑efficient. Experiments fine‑tuning Qwen2.5‑1.5B‑Instruct demonstrate that a moderate density of states balances accuracy and robustness, and that real‑world models like Qwen3 retain most of their performance even when attention is limited to a small sliding window.

By Amrut Nadgir, Pratik Chaudhari, Vijay Balasubramanian
llmsfine-tuning
More like this →