← Back to all news
arXiv Machine Learning By Chenxiao Yang, Zhiyuan Li, David McAllester, Nathan Srebro

Recursive Language Models Generalize Out of Domain

Read the original on arXiv Machine Learning →

The paper investigates how restricting a language model’s view to isolated subtask contexts—via recursive language models—affects learning compared to standard chain‑of‑thought (CoT) approaches. While CoT can simulate recursive reasoning and offers similar in‑distribution performance, it tends to exploit contextual shortcuts that fail out of distribution; recursive models avoid this by isolating context, thereby preventing such shortcuts. The study shows that merely covering the correct rule is insufficient for robust reasoning beyond the training distribution.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Computation and Language
Sep 4

LLMs Learn Better In-Context from Rules than from Examples

The paper investigates how large language models learn new tasks in-context, comparing rule-based instruction following to example-based few-shot prompting across five diverse tasks. Results show that models generally learn more reliably from rule descriptions than from examples alone, and adding more examples does not consistently improve performance. Instruction tuning further enhances rule-based learning while preserving example-based capabilities, with rule advantages being strongest for algebraic tasks and weaker for tasks requiring distributional sensitivity or parametric knowledge.

By Xiang Fu, Seungmin Cho, Yukyung Lee, Najoung Kim
llmsfine-tuning
More like this →
arXiv Machine Learning
Jun 29

Learning to Reason with Curriculum II: Compositional Generalization

arXiv:2606. 27721v1 Announce Type: new Abstract: Compositional generalization, the ability to solve complex problems by combining solutions to simpler sub-problems, is a fundamental capability of both natural and artificial intelligence, and a key mechanism underlying chain-of-thought reasoning.

By Nived Rajaraman, Audrey Huang, Miroslav Dudik, Robert Schapire, Dylan Foster, Akshay Krishnamurthy
reinforcement-learningfine-tuning
More like this →
arXiv AI
Jun 26

Where Do CoT Training Gains Land in LLM based Agents?

arXiv:2606. 26935v1 Announce Type: new Abstract: Chain-of-thought (CoT) reasoning is widely used in language-model agents, but prior work has shown that verbalized CoT is not always faithful and may instead reflect post-hoc reasoning, which means the model already knows the answer before reasoning.

By Jingyu Liu, Zhiwen Wang, Yuxin Jing, Huanyu Zhou, Yong Liu
llmsagents
More like this →