The paper investigates why neural networks trained on modular addition tasks develop Fourier-structured representations that enable exact generalization. Using probability signatures, the authors show that leading gradient interactions are cyclic shift operators, which are diagonalized by the discrete Fourier transform, leading to decoupled Fourier-mode dynamics that explain Fourier sparsity, frequency matching, and phase alignment. The framework also clarifies how label noise can accelerate early loss decrease by increasing conditional label collisions, and it is demonstrated to predict frequency behavior in other operators such as XOR.
By Yunji Wang, Junjie Yao, Linyu Liu, Pinyan Lu, Zhi-Qin John Xu
arXiv:2607. 15693v1 Announce Type: cross Abstract: We describe a model of perceptual inference in primary visual cortex (V1) equivalent to a minimal diffusion model whose function can be readily understood from its parameters.
By Zeyu Yun, Alexander Belsten, Dasheng Bi, Zahra Kadkhodaie, Yubei Chen, Bruno A. Olshausen
Nemotron‑Labs‑Diffusion‑Uni (NLD‑Uni) is a unified discrete diffusion large language model designed for visual understanding and generation. It introduces a cross‑paradigm on‑policy distillation framework that transfers capabilities from autoregressive VLMs and continuous flow models into a single DLLM, and a progressive self‑refining diffusion process that corrupts tokens gradually during training. Experiments show NLD‑Uni outperforms other unified DLLMs on tasks such as visual question answering, multi‑modal reasoning, object grounding, text‑to‑image generation, and image editing.
By Shufan Li, Hanrong Ye, Greg Heinrich, Yonggan Fu, Haotian Zhang, Sifei Liu, Aditya Grover, Pavlo Molchanov
The paper argues that neural networks can exhibit modular organization even when individual neurons are not specialized. It introduces a nested hierarchy of modularity—unit-level, orthogonal, and linear—and shows that many vision, language, and RNN models lack unit-level modularity but still display clear functional modularity at the subspace level. The authors demonstrate that these subspaces can be causally validated through ablation and align with brain data, revealing distinct content/style or syntax/semantics separations.
By Shreya Saha, Benjamin Bergen, Meenakshi Khosla
arXiv:2606. 03645v1 Announce Type: cross Abstract: Large Language Models exhibit paradoxical fragility in fundamental arithmetic, implying a disconnect between internal computation and discrete output.
By Liuyuan Wen, Xun Zhu, Lihao Huang, Wenbin Li, Yang Gao
arXiv:2603.23784v1 Announce Type: cross
Abstract: Grokking-the phenomenon where validation accuracy of neural networks on modular addition of two integers rises long after training data has been memo...
By Anand Swaroop