arXiv Machine Learning

A Thermodynamic Theory of Learning Part II: History-Dependent Reachability and Continual Learning

arXiv AI
Oct 28

Beyond Parameter Protection: Fisher-Shaped EWC for Replay-Free Continual Learning

The paper introduces Fisher-Shaped EWC (FS‑EWC), a replay‑free continual learning method that extends Elastic Weight Consolidation by controlling Fisher sensitivity over a neighborhood of future updates. FS‑EWC minimizes the worst‑case empirical Fisher information trace during task learning, favoring solutions robust to bounded parameter displacement, and uses trajectory‑based model merging to construct the retained solution from multiple optimization states. Experiments on several benchmarks demonstrate that FS‑EWC outperforms standard EWC, achieving a better stability–plasticity balance and higher tolerance to parameter drift.

By Liying Wang, Simone Magistri, Joost van de Weijer
arXiv Machine Learning
Oct 8

Identifiability of a dissipative knowledge-dynamics model: exact recovery under designed excitation, degeneration on observational data

The paper presents a nonlinear dissipative dynamical model of human learning, featuring a concept‑transfer matrix, per‑concept forgetting rates, and a saturating practice‑response gain. It proves a structural identifiability theorem that guarantees exact parameter recovery under explicit excitation conditions, provides closed‑form recovery for the two‑concept case, and introduces a fast, L‑stable numerical scheme enabling estimation on large learner cohorts. Empirical results show exact recovery on synthetic data with designed excitation, but failure to recover on large observational benchmarks, illustrating a sharp boundary between identifiable and unidentifiable regimes and offering a validation protocol for interpretability claims.

By Arman Kostanian, Armen Beklaryan
arXiv Machine Learning
Oct 3

Learning Goal-Reaching Quasimetric Geometry From Finite-Time Reachability

The paper introduces ReQRL, a method for learning quasimetric geometry in goal-conditioned reinforcement learning by constraining the critic’s value gradients with finite-horizon reachability. It decouples dynamical reachability from boundary geometry, estimating both from data using state-constrained optimal control principles. Experiments on OGBench show that ReQRL matches or surpasses existing quasimetric and offline GCRL approaches.

By Daisuke Yamada, Travis Pence, Vikas Singh
Hugging Face Trending Papers
Oct 27

Beyond Parameter Protection: Fisher-Shaped EWC for Replay-Free Continual Learning

The paper introduces Fisher-Shaped EWC (FS‑EWC), a method that extends Elastic Weight Consolidation by controlling Fisher sensitivity over a neighborhood of potential future updates. FS‑EWC minimizes the worst‑case empirical Fisher information trace during task learning, favoring solutions less sensitive to bounded future displacement, and uses trajectory‑based model merging to construct the retained solution. Experiments on multiple benchmarks show that FS‑EWC outperforms standard EWC, achieving a better stability–plasticity balance and greater tolerance to parameter drift.

arXiv Machine Learning
Sep 15

Learning to Solve Stochastic Controls with Unknown Drifts and Running Rewards: Theory, Algorithms and Convergence

The paper investigates continuous‑time stochastic control problems with unknown drift and running reward functions, using an exploratory reinforcement learning framework that incorporates relaxed controls and entropy regularization. It develops policy‑iteration algorithms based on probabilistic representations of the optimal value function and its gradient, proving convergence and demonstrating performance through numerical examples. The study also extends to a special case with control‑dependent diffusion, requiring a Hessian representation.

By Jin Ma, Gaozhan Wang, Jianfeng Zhang, Xunyu Zhou
arXiv Machine Learning
Oct 8

An Informational Curse of Horizon in Goal-Conditioned Policy Learning

The paper identifies an informational curse of horizon in goal-conditioned policy learning, showing that longer goal relabeling horizons during training can hurt policy generalization and performance. Controlled experiments with oracle planners separate training horizons from test horizons, revealing that behavioral cloning policies degrade with longer horizons while reinforcement learning mitigates this effect. The authors attribute this to reduced conditional mutual information between actions and hindsight-relabeled goals, and demonstrate that distilling short-horizon policy Jacobians into long-horizon policies improves performance, especially in combinatorial manipulation tasks.

By John L. Zhou, Yuxuan Dong, Jonathan C. Kao