The paper investigates why neural networks trained on modular addition tasks develop Fourier-structured representations that enable exact generalization. Using probability signatures, the authors show that leading gradient interactions are cyclic shift operators, which are diagonalized by the discrete Fourier transform, leading to decoupled Fourier-mode dynamics that explain Fourier sparsity, frequency matching, and phase alignment. The framework also clarifies how label noise can accelerate early loss decrease by increasing conditional label collisions, and it is demonstrated to predict frequency behavior in other operators such as XOR.
By Yunji Wang, Junjie Yao, Linyu Liu, Pinyan Lu, Zhi-Qin John Xu
arXiv:2607. 15693v1 Announce Type: cross Abstract: We describe a model of perceptual inference in primary visual cortex (V1) equivalent to a minimal diffusion model whose function can be readily understood from its parameters.
By Zeyu Yun, Alexander Belsten, Dasheng Bi, Zahra Kadkhodaie, Yubei Chen, Bruno A. Olshausen
Nemotron‑Labs‑Diffusion‑Uni (NLD‑Uni) is a unified discrete diffusion large language model designed for visual understanding and generation. It introduces a cross‑paradigm on‑policy distillation framework that transfers capabilities from autoregressive VLMs and continuous flow models into a single DLLM, and a progressive self‑refining diffusion process that corrupts tokens gradually during training. Experiments show NLD‑Uni outperforms other unified DLLMs on tasks such as visual question answering, multi‑modal reasoning, object grounding, text‑to‑image generation, and image editing.
By Shufan Li, Hanrong Ye, Greg Heinrich, Yonggan Fu, Haotian Zhang, Sifei Liu, Aditya Grover, Pavlo Molchanov
The paper argues that neural networks can exhibit modular organization even when individual neurons are not specialized. It introduces a nested hierarchy of modularity—unit-level, orthogonal, and linear—and shows that many vision, language, and RNN models lack unit-level modularity but still display clear functional modularity at the subspace level. The authors demonstrate that these subspaces can be causally validated through ablation and align with brain data, revealing distinct content/style or syntax/semantics separations.
By Shreya Saha, Benjamin Bergen, Meenakshi Khosla
arXiv:2606. 03645v1 Announce Type: cross Abstract: Large Language Models exhibit paradoxical fragility in fundamental arithmetic, implying a disconnect between internal computation and discrete output.
By Liuyuan Wen, Xun Zhu, Lihao Huang, Wenbin Li, Yang Gao
arXiv:2603.23784v1 Announce Type: cross
Abstract: Grokking-the phenomenon where validation accuracy of neural networks on modular addition of two integers rises long after training data has been memo...
By Anand Swaroop
While unlearning seeks to negate undesired capabilities acquired through learning, little research has examined how the way models learn shapes their subsequent unlearning. In this paper, we investiga...
arXiv:2609.39648v1 Announce Type: cross
Abstract: Diffusion models are typically viewed as stochastic processes that transform noise into data. We take a complementary perspective: a diffusion model...
By Cristina L\'opez Amado, Marco Fumero, Francesco Locatello
arXiv:2610.28086v1 Announce Type: cross
Abstract: Real-time video generation, interactive world models, and on-device image synthesis are almost a reality. Diffusion Transformers (DiT) achieve excell...
By Jarek Liesen, Matthew T. Jackson, Benjamin Ellis, Shimon Whiteson, Jakob N. Foerster
arXiv:2410. 24050v3 Announce Type: replace Abstract: Large-scale pretraining of transformers has been central to the success of foundation models.
By Ambroise Odonnat, Wassim Bouaziz, Vivien Cabannes
arXiv:2603. 28762v2 Announce Type: replace-cross Abstract: Modern Text-to-Image (T2I) diffusion models have achieved remarkable semantic alignment, yet they often suffer from a significant lack of variety, converging on a narrow set of visual solutions for any given prompt.
By Omer Dahary, Benaya Koren, Daniel Garibi, Daniel Cohen-Or
Mode Dynamics is a framework that links coordinated parameter motion during neural network training to changes in model outputs. By decomposing recorded parameter trajectories into collective modes and mapping each mode to its state‑dependent output effect, the framework uses decomposition, reconstruction, and concentration mechanisms to analyze how behavior emerges from parameter updates. Experiments on CIFAR‑100 autoencoding and a Pythia‑70M pretraining trajectory show that a small set of leading modes can reconstruct behavior, that induced effects depend on the training path, and that modal complexity varies with task and data.