← Back to all news
arXiv Machine Learning By Micha{\l} Kozyra, Michael Bronstein

Interfaces and Modules: A Scalable Representation for Weight-Space Learning across Architectures

Read the original on arXiv Machine Learning →

The paper introduces the Universal Weight Encoder (UWE), a scalable representation for weight-space learning that uses a typed graph of interfaces and modules to respect parameter symmetries across diverse neural network architectures. UWE combines local equivariant updates with global attention over invariant interface summaries, enabling a single encoder to process multiple supported architectures efficiently. Experiments show UWE matches or surpasses architecture‑specific state‑of‑the‑art methods on standard benchmarks and excels on a large zoo of 1,200 transformers, ranking unseen architectures accurately while using significantly less memory.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Oct 3

CrossGMN: Graph Metanetworks for Cross-Architecture Weight-Space Transformations

CrossGMN introduces a graph metanetwork that processes a trained source network and an initialized target network simultaneously, enabling equivariant cross‑architecture weight‑space transformations. By preserving symmetry through cross‑network message passing, CrossGMN can refine target network initializations while remaining invariant to source permutations and equivariant to target permutations. Experiments demonstrate that CrossGMN accelerates knowledge distillation, transfers across datasets without retraining, and unifies compression from diverse source architectures into a common target architecture.

By Adir Dayan, Yam Eitan, Haggai Maron
llmscomputer-visionefficiency
More like this →