arXiv Machine Learning

Unified dynamics of training, learning, inference, and reinforcement in neural networks

The paper reports nine nanoGPT experiments that uncover relationships among training, learning, inference, and reinforcement in neural networks. It shows that updates trigger state‑dependent responses, learning reorganizes functional support, and inference uses this support without altering it. A predictor derived from the fifth experiment accurately forecasts post‑update capability transitions, and the authors extend these findings to other architectures, demonstrating that balanced feedback can restore lost skills and form a unified dynamical framework for neural‑network behavior.

arXiv AI
Aug 7

Continual Learning in Transition

arXiv:2608. 06216v1 Announce Type: cross Abstract: Classical continual learning (CL) has primarily focused on enabling models to update and retain knowledge through parameter-centric mechanisms, e.

By Zhiyan Hou, Dan Zhang, Tao Feng, Liyuan Wang, Wei Li, Xiangzhao Hao, Hongyan An, Junfeng Fang, Haokai Ma, Zhaohui Xu, Haiyun Guo, Jinqiao Wang, Tat-Seng Chua
arXiv Computation and Language
Sep 28

Recursive Self-Improvement via On-Policy Distillation for Reasoning

The paper introduces a recursive self-improvement framework for language models that replaces an external teacher with a frozen copy of the student, enabling dynamic co-evolution (DCE) and self-refined concise learning (SRCL). DCE allows the privileged teacher to evolve alongside the student, while SRCL trains on shorter, verified rewrites to reduce verbosity. Experiments show that the combined DCE+SRCL approach outperforms traditional on‑policy self‑distillation across multiple model sizes and math benchmarks, achieving significant accuracy gains and shorter outputs.

By Shangjian Yin, Zehao Zhao, Kavosh Asadi, Rui Liu, Yuchen Lu, Shike Mei, Hang Cui, Luke Simon, Zhouxing Shi, Hamed Firooz
arXiv Machine Learning
Sep 25

Intrinsic-Extrinsic Coupling in Learning Dynamics

The paper introduces a framework for intrinsic‑extrinsic coupling in learning dynamics, defining it via a continuation‑conditioned value of a constrained learning‑state intervention and observation‑relative fibers. It presents an executable finite‑frame classifier‑head that protects current logits while repairing historical margins, and distinguishes local admissibility, intervention value, and complete‑policy performance. Experiments on CLINC‑derived class‑incremental tasks, output distillation with RoBERTa, and SGDW dynamics demonstrate that coupling can produce both positive and negative interactions, and that coordinated content controls can match or exceed development gains while guided allocation reduces cross‑entropy loss compared to standard replay.

By Qinyou Wang
Hugging Face Trending Papers
Sep 24

Intrinsic-Extrinsic Coupling in Learning Dynamics

The paper introduces a framework for intrinsic‑extrinsic coupling in learning dynamics, where a learner’s current observations do not solely dictate its future training responses. It formalizes this coupling through a continuation‑conditioned value of a constrained learning‑state intervention and employs an executable finite‑frame classifier‑head to protect current logits while adjusting historical margins. Experiments across CLINC‑derived class‑incremental settings, output distillation with RoBERTa, and SGDW dynamics reveal that coupling can produce both positive and negative interactions, and that coordinated interventions can match or exceed development gains while reducing cross‑entropy loss compared to standard replay.

arXiv Machine Learning
Jul 1

Predictable GRPO: A Closed-Form Model of Training Dynamics

arXiv:2606. 30789v1 Announce Type: new Abstract: Group Relative Policy Optimization (GRPO) has become a standard tool for improving the reasoning ability of large language models, yet its training dynamics are still described empirically: reward trajectories are fit with low-parameter functional forms whose constants carry no mechanistic meaning, and hyperparameter choices remain a matter of trial and error.

By Rajat Ghosh, Datta Nimmaturi, Aryan Singhal, Vaishnavi Bhargava, Henry Wong, Johnu George, Debojyoti Dutta
arXiv AI
Oct 19

Interpretable experiential learning based on state history and global feedback

The paper introduces an interpretable experiential learning model that uses state history and global feedback to learn a behavioral model represented as a transition graph. Transitions in this graph are annotated with utility values and evidence counts, enabling the model to function in resource-constrained settings. The authors evaluate the approach on the OpenAI Gym Atari Breakout benchmark, showing performance comparable to certain neural network-based methods.

By Anton Kolonin