arXiv:2608. 04451v1 Announce Type: cross Abstract: We give a negative solution to MAIS-O60.
By Gautam Neelakantan Memana
arXiv:2610. 11475v1 Announce Type: new Abstract: Population gradient flow is a common tool for reasoning about how neural networks adapt, including after pretraining.
By Ruoyu Zhao, Mingxuan Zhang, Jianbo Dai, Jiaqi Wu, Chenyu Zhu, Tong Che
Population gradient flow is a common tool for reasoning about how neural networks adapt, including after pretraining. We show that it can mispredict finite-batch stochastic gradient descent (SGD) qualitatively, and we trace the discrepancy to a specific mechanism.
The paper introduces a new method called uniform‑phase initialization for deep neural networks with sine activations, eliminating the need for the Central Limit Theorem and fully decoupling layers. This approach avoids distributional approximation errors and coupling between layers, leading to stable weight initialization. Experiments show that models using this initialization outperform state‑of‑the‑art methods on image and audio fitting tasks and remain competitive without tuning, while also supporting μP width scaling.
By Simon Kuang, Kyle Chickering, Xinfan Lin
arXiv:2609. 12996v1 Announce Type: new Abstract: We prove a full Adam theorem for spectral heavy-tail onset in a closed Gaussian Stein-Hermite teacher-student state-evolution model.
By Zongmin Liu
arXiv:2610. 31663v1 Announce Type: cross Abstract: Constructing accurate and computationally tractable approximation spaces is a fundamental task in scientific computing.
By Xin He, Huan Lao, Xiliang Lu, Shijun Zhang