← Back to all news
arXiv Computation and Language By Hadeel Al-Negheimish, Jasna Ilieva, Yoon Kim

Probing for Long-Horizon Deductive Reasoning Capabilities in Language Models with Prolog

Read the original on arXiv Computation and Language →

The paper introduces ProloNg, a synthetic benchmark designed to test long-horizon deductive reasoning in language models using Prolog. It evaluates eight reasoning models from five leading LLM families on problems with reasoning depths up to 22 and context lengths of 62,000 tokens. Results show that model performance drops sharply as reasoning depth increases, with most models performing near chance beyond a depth of 10.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv AI
Sep 7

Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Models

The paper surveys efficient reasoning in large language models, contrasting fast intuitive (System 1) and slow deep (System 2) reasoning. It analyzes why System 2 is computationally costly yet more accurate, and why System 1 is efficient but less effective. The survey covers causes of inefficiency, patterns of reasoning behavior, and potential solutions to balance performance and computational budgets, offering actionable insights and an open‑source repository for ongoing research.

By Rui Wang, Hongru Wang, Boyang Xue, Jianhui Pang, Shudong Liu, Yi Chen, Jiahao Qiu, Derek Fai Wong, Heng Ji, Kam-Fai Wong
llms
More like this →
arXiv AI
Jun 15

Fractured Chain-of-Thought Reasoning

arXiv:2505. 12992v4 Announce Type: replace-cross Abstract: Inference-time scaling techniques have significantly bolstered the reasoning capabilities of large language models (LLMs) by harnessing additional computational effort at inference without retraining.

By Baohao Liao, Hanze Dong, Yuhui Xu, Doyen Sahoo, Christof Monz, Junnan Li, Caiming Xiong
llmsbenchmarks
More like this →
Hugging Face Trending Papers
Jul 9

Understanding Axes of Difficulty For Long Context Tasks Via PredicateLongBench

Large language models (LLMs) have demonstrated rapidly improving long-context capabilities, prompting a wave of benchmarks designed to evaluate them. However, existing long-context evaluations - from Needle-in-a-Haystack (NIAH) tests to more recent multi-hop reasoning and summarization tasks - predominantly measure average-case performance, and many are either saturated or lack robustness.

llmsnlpbenchmarks
More like this →