The paper reviews Cobweb, a computational model of categorization and concept formation, and extends it to include chunks and their acquisition. It introduces rellis/, an implementation that applies this unified theory to learning context-free grammars, demonstrating the system’s ability to represent syntactic knowledge, parse and generate sentences, and learn compositional structures from sample parses. The authors discuss related work on concepts and chunks and suggest directions for future research.
By Karthik Singaravadivelan, Pat Langley
arXiv:2507. 01414v2 Announce Type: replace Abstract: We introduce a new family of toy problems that combine features of linear-regression-style continuous in-context learning (ICL) with discrete associative recall.
By Sultan Daniels, Dylan Davis, Dhruv Gautam, Wentinn Liao, Gireeja Ranade, Anant Sahai
arXiv:2606. 27721v1 Announce Type: new Abstract: Compositional generalization, the ability to solve complex problems by combining solutions to simpler sub-problems, is a fundamental capability of both natural and artificial intelligence, and a key mechanism underlying chain-of-thought reasoning.
By Nived Rajaraman, Audrey Huang, Miroslav Dudik, Robert Schapire, Dylan Foster, Akshay Krishnamurthy
arXiv:2603.21676v2 Announce Type: replace-cross
Abstract: Standard Transformers have a fixed computational depth, limiting their ability to generalize to tasks that require variable-depth reasoning....
By Hung-Hsuan Chen
The paper investigates how different forms of compressed chain‑of‑thought (CoT) reasoning—Explicit, Composed, and Implicit—affect large language model (LLM) performance after supervised fine‑tuning (SFT). Using a synthetic compositional reasoning task, the authors show that coarser CoT requires more SFT data, that Composed and Implicit CoT benefit more from data scaling (with Composed also benefiting from repetition), and that reinforcement learning with verifiable rewards (RLVR) can decompose compressed steps learned during SFT. Additionally, unidirectional CoT ordering improves generalization on longer sequential tasks.
By Kohsei Matsutani, Gouki Minegishi, Takeshi Kojima, Yusuke Iwasawa, Yutaka Matsuo
arXiv:2506. 20699v2 Announce Type: replace Abstract: Learning in non-stationary and multi-context environments requires more than ordinary within-task generalization.
By Xin Li
Relational BabyLM is a decoder‑only Transformer that replaces standard self‑attention with a Dual Attention Transformer (DAT) to separate object‑level lexical features from structural/relational information. The model incorporates a Next‑Latent Prediction objective to compress history into a dense belief state and introduces a RoPE‑based symbol‑retrieval mechanism. On the BabyLM 2026 challenge, the best model ranks 6th overall and 3rd on the NLP‑task subset, outperforming GPT‑2 on most benchmarks and achieving the highest EWoK score among strict‑track entries.
By Adrian Brasoveanu, Ece Takmaz, Jakub Dotla\v{c}il
arXiv:2510. 13554v2 Announce Type: replace-cross Abstract: The reasoning pattern of Large language models (LLMs) remains opaque, and reinforcement learning (RL) typically applies uniform credit across an entire generation, blurring the distinction between pivotal and routine steps.
By Yang Li, Zhichen Dong, Yuhan Sun, Weixun Wang, Shaopan Xiong, Yijia Luo, Jiashun Liu, Han Lu, Jiamang Wang, Wenbo Su, Bo Zheng, Junchi Yan
arXiv:2604. 01577v3 Announce Type: replace-cross Abstract: We study out of distribution generalization in streaming tasks where models are trained on short sequences but must operate over much longer, unknown horizons under bounded memory.
By Shota Takashiro, Masanori Koyama, Takeru Miyato, Yusuke Iwasawa, Yutaka Matsuo, Kohei Hayashi
arXiv:2604. 07822v2 Announce Type: replace-cross Abstract: We study implicit reasoning, i.
By Harsh Kohli, Srinivasan Parthasarathy, Huan Sun, Yuekun Yao
arXiv:2510.01030v2 Announce Type: replace
Abstract: The human ability to translate diverse perceptual and linguistic inputs into structured behavior has been thought to rest on learning robust repres...
By Zach Studdiford, Timothy T. Rogers, Kushin Mukherjee, Siddharth Suresh
The paper investigates how LLaMA 3.1‑8B models numerical sequence patterns, focusing on time‑series prediction. By designing a task that requires detecting structural cues—specifically first differences in a sequence—the authors show that the model performs well and internally computes and stores these differences. Probing and activation‑patching experiments reveal that LLaMA retrieves and applies the first‑difference via an induction‑like circuit, marking one of the first demonstrations of concept induction in large language models.
By Rahul Chowdhury, Timothy A Rupprecht, Senhao Cao, Jiahao Liu, Octavia Camps, David Bau, Pu Zhao, Yanzhi Wang