← Back to all news
arXiv AI

Knowledge transfer between simple discrete worlds

Related stories

arXiv Machine Learning
Oct 8

The Dichotomy Between Pattern Recognition and Step-by-Step Reasoning

The paper argues that pattern recognition and step‑by‑step reasoning lie on a spectrum, with large language models (LLMs) learning the latter when the next token depends on only a few preceding tokens. It formalises reasoning traces as paths on a De Bruijn graph, showing that the number of edges is far smaller than the number of possible traces, making step‑by‑step reasoning sample‑efficient. Experiments fine‑tuning Qwen2.5‑1.5B‑Instruct demonstrate that a moderate density of states balances accuracy and robustness, and that real‑world models like Qwen3 retain most of their performance even when attention is limited to a small sliding window.

By Amrut Nadgir, Pratik Chaudhari, Vijay Balasubramanian
llmsfine-tuning
More like this →
arXiv AI
Oct 15

Tiny Recursive Message Passing

Tiny Recursive Message Passing applies recursive transformers and deep supervision to message passing neural networks, achieving high efficiency. The approach yields about 99% test accuracy on Sudoku-Extreme with only 16k parameters, training quickly on modern GPUs or CPUs. It also demonstrates zero‑shot transfer to larger problem instances, outperforming popular neural baselines on N‑queens and Boolean satisfiability with fewer parameters.

By Tommaso Salvatori, Matteo Forasassi, Alexis Toumi, Giovanni de Felice
llms
More like this →
arXiv Computation and Language
Aug 25

Beyond Factual Knowledge: Benchmarking and Learning Step-Level Procedural Rule Reasoning in Large Language Models

arXiv:2608.22753v1 Announce Type: new Abstract: Large language models (LLMs) excel at text understanding and generation, yet still struggle to reliably understand and apply externally provided proced...

By Bohan Yu, Pengfei Cao, Chen Han, Chenxi Zhou, Zhiheng Zhang, Zhiyang Xie, Wenhao Teng, Xiangwen Liao, Jun Zhao, Kang Liu
llmsnlpefficiencybenchmarks
More like this →
arXiv Computation and Language
Aug 28

Why Knowing Both Hops Is Not Enough: Understanding Two-Hop Generalization in Language Models

Large language models can solve complex multi‑hop tasks but often fail on simple two‑hop queries, even when each hop is individually correct. In a controlled symbolic setting, the authors find that models generalize reliably when the second hop follows the training distribution, but always fail when it deviates. Mechanistic analysis shows that successful generalization relies on consistent intermediate representations across contexts, whereas failures arise from a mismatch between lower‑layer representation construction and upper‑layer mapping to outputs. The study proposes a recurrent‑style training strategy that improves out‑of‑distribution two‑hop generalization.

By Zili Zhang, Yilin Wang, Heng Wang, Herun Wan, Minnan Luo
llms
More like this →
arXiv AI
Sep 2

Flow Reasoning Models: Turning Flows Into Efficient Recurrent Reasoners

Flow Reasoning Models (FRMs) are a new framework that turns continuous flow models into efficient recurrent reasoners for structured tasks. By self‑conditioning a flow model on its own past outputs, FRMs iteratively refine solutions, allowing parallel decision making and revision. The authors introduce Fixed‑Point Forcing (FPF) to mitigate exposure bias at deeper recursion, and report near‑perfect solve rates on Sudoku‑Extreme, Zebra, and Maze‑Unique, outperforming existing masked‑diffusion and specialized baselines while using far fewer inference FLOPs.

By Alec Helbling, Andrey Bryutkin, Mauro Martino, Duen Horng Chau, Nima Dehmamy, Hendrik Strobelt
diffusionsafety
More like this →