← Back to all news
arXiv Machine Learning By Guy Hadad, Haggai Roitman, Moshe Eliasof, Bracha Shapira

Module-Aware Optimization for Graph Neural Networks

Read the original on arXiv Machine Learning →

The paper introduces GraphMuon, a graph-aware optimizer that extends the matrix-aware Muon optimizer by incorporating the covariance structure of graph‑propagated layer inputs. GraphMuon performs orthogonalization in a metric defined by these propagated representations and normalizes step sizes based on the update’s effect on the layer output, thereby mitigating graph‑induced update amplification. Empirical results show that GraphMuon achieves 1.32×–2.42× faster validation‑loss time‑to‑target compared to AdamW across 15 dataset‑architecture configurations on large‑scale OGB benchmarks, while maintaining competitive predictive performance.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Oct 12

From SGD to Muon: Adaptive Optimization via Schatten-p Norms

The paper introduces a data‑driven criterion that dynamically selects optimal Linear Minimization Oracle (LMO) geometries for each layer of a deep neural network, bridging the gap between SGD and Muon updates. By deriving the criterion from gradient and activation statistics via a single‑step random feature regression surrogate, the authors enable a design space that interpolates between existing optimizers, including SGD, Muon, Adam, and MuAdam. The adaptive approach incurs only about a 3% runtime overhead and demonstrates competitive or superior performance compared to the best static optimizer across three training scenarios.

By Thomas Massena (DTIPG - SNCF, IRIT), Corentin Friedrich (IRIT), Mathieu Serrurier (IRIT)
More like this →
arXiv Machine Learning
Aug 19

Asynchronous Message Passing for Addressing Oversquashing in Graph Neural Networks

The paper introduces an asynchronous message‑passing framework for Graph Neural Networks to mitigate oversquashing, a problem where distant nodes cannot effectively communicate due to structural bottlenecks. Unlike conventional synchronous updates, the method updates a centrality‑guided batch of nodes at each layer, allowing information to propagate sequentially and reducing the need for increased channel capacity. Experiments on six standard and two long‑range graph classification benchmarks show notable performance gains, including 5 % improvement on REDDIT‑BINARY and 4 % on Peptides‑struct.

By Kushal Bose, Swagatam Das
benchmarkssafety
More like this →