Hugging Face Trending Papers

Autonomous Continual Learning and Knowledge Transfer via Cortex-Inspired Neural Assemblies

Read the original on Hugging Face Trending Papers →

The paper introduces Functional Task Networks (FTNs), a continual learning framework inspired by mammalian neocortex. FTNs use a distributed, stochastic mixture of experts with a sparse binary routing mask that is computed at inference time from a small support set of input-output pairs. This per-batch router reconfiguration, combined with mask-overlap-filtered replay, allows unrelated tasks to use separate assemblies while related tasks share components, achieving strong retention and transfer performance on block‑sequentially trained tasks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

arXiv AI
Jul 22

Soft-TransFormers for Continual Learning

arXiv:2411. 16073v4 Announce Type: replace-cross Abstract: Inspired by the Well-initialized Lottery Ticket Hypothesis (WLTH), we introduce Soft-TransFormers (Soft-TF), a continual learning framework that adapts a frozen pre-trained Transformer through task-specific soft subnetworks: real-valued multiplicative masks over the query, key, value, and output projections of selected self-attention layers.

By Haeyong Kang, Chang D. Yoo
Hugging Face Trending Papers
Jun 29

Neural Subspace Reallocation: Continual Learning as Retrieval-Based Subspace Memory Management

We introduce Neural Subspace Reallocation (NSR), which reframes continual learning as memory management over parameter subspaces. Instead of treating Low-Rank Adaptation (LoRA) modules as disposable per-task adapters, NSR manages them as compressible, retrievable memory units on a frozen backbone through a recurring cycle: (1) compress learned LoRAs via SVD, (2) reserve them in a TaskKnowledgeBank, (3) recall related past LoRAs by embedding similarity to warm-start new or returning tasks, and (4) reallocate the active subspace accordingly, with distillation protecting prior tasks.

arXiv Machine Learning
Oct 28

FedSEAM: Federated Continual Learning of LLMs via Subspace Expert Agreement Merging

FedSEAM is a federated continual learning framework that trains large language models across multiple clients without sharing data, using sparse mixture-of-experts to adapt selected parameters on a frozen backbone. It introduces task-aware tile selection for new tasks and a subspace-based compatibility test to decide which expert tiles to reuse or replace. On the server, Subspace Expert Agreement Merging (SEAM) aligns client updates only along agreed subspaces, while additional protections mitigate routing drift and merge erasure, preserving prior knowledge during training.

By Fatema Siddika, Abhijit Chunduru, Zilinghan Li, Ali Jannesari, Ravi Madduri