Hugging Face Trending Papers August 3, 2026

Measuring in-context algorithmic reasoning in language models against an exact Bayes-optimal standard

Read the original on Hugging Face Trending Papers →

Whether large language models perform genuine algorithmic reasoning or mere pattern completion is hard to test, because most benchmarks lack a ground truth for correct inductive inference. We introduce F-ICL, an in-context-learning benchmark that supplies one exactly.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

arXiv Machine Learning

Aug 4

Measuring in-context algorithmic reasoning in language models against an exact Bayes-optimal standard

arXiv:2608. 01575v1 Announce Type: new Abstract: Whether large language models perform genuine algorithmic reasoning or mere pattern completion is hard to test, because most benchmarks lack a ground truth for correct inductive inference.

By Hector Zenil, Luan Ozelim

llms benchmarks safety

arXiv AI

Aug 25

Measuring in-context algorithmic reasoning in language models against an exact Bayes-optimal reference

The paper introduces F-ICL, a benchmark that measures in‑context algorithmic reasoning in language models by exhaustively enumerating 86 million valid programs of length ≤13 on a Turing‑complete machine and computing the exact posterior under a bounded Levin–Solomonoff prior. Unlike typical benchmarks, F‑ICL provides a distributional reference rather than just answers, allowing the evaluation of models’ inductive priors. Across 105 configurations of models ranging from 0.8 B to 675 B parameters, models achieve up to 92 % accuracy, yet many still deviate from the Bayes‑optimal reference, and the study derives theoretical bounds on cumulative loss for predictors with positive prior weight on the reference.

By Luan Ozelim, Hector Zenil

llms benchmarks safety

arXiv AI

Jun 2

Off-the-Shelf LLMs as Process Scorers: Training-Free Alternative to PRMs for Mathematical Reasoning

arXiv:2606. 01682v1 Announce Type: cross Abstract: Selecting the best response from multiple small-model samples using a stronger scorer is a simple inference-time strategy, but fails when the small model has already committed to incorrect reasoning paths.

By Atoosa Chegini, Soheil Feizi

llms reinforcement-learning benchmarks safety

Hugging Face Trending Papers

Jun 1

Off-the-Shelf LLMs as Process Scorers: Training-Free Alternative to PRMs for Mathematical Reasoning

Selecting the best response from multiple small-model samples using a stronger scorer is a simple inference-time strategy, but fails when the small model has already committed to incorrect reasoning paths. PRM guided search avoids this by scoring candidate continuations during generation, but requires a reward model trained with step-level labels.

llms reinforcement-learning benchmarks safety

arXiv AI

Sep 1

DIP: Dynamic In-Context Planner For Diffusion Language Models

arXiv:2601.03199v2 Announce Type: replace-cross Abstract: Diffusion language models (DLMs) have shown strong potential for general natural language tasks with in-context examples. Existing In-Context...

By Yang Li, Han Meng, Chenan Wang, Zhenyu Bi, Xuan Wang, Haipeng Chen

llms diffusion benchmarks

arXiv Computation and Language

Sep 2

Compile, Don't Memorize: A Context Compilation Architecture (CCA) for In-Context Learning

arXiv:2609.00759v1 Announce Type: new Abstract: Large language models (LLMs) increasingly handle in-context learning (ICL) tasks where a long, novel context defines the rules, knowledge, and output s...

By Jinhu Qi, Minda Hu, Wentao Zhang, Weiqiang Jin, Yanyu Chen, Junli Wang, Irwin King

llms agents reinforcement-learning benchmarks

One email a morning, machine-written

Related stories

Measuring in-context algorithmic reasoning in language models against an exact Bayes-optimal standard

Measuring in-context algorithmic reasoning in language models against an exact Bayes-optimal reference

Off-the-Shelf LLMs as Process Scorers: Training-Free Alternative to PRMs for Mathematical Reasoning

Off-the-Shelf LLMs as Process Scorers: Training-Free Alternative to PRMs for Mathematical Reasoning

DIP: Dynamic In-Context Planner For Diffusion Language Models

Compile, Don't Memorize: A Context Compilation Architecture (CCA) for In-Context Learning