arXiv:2606. 06624v1 Announce Type: new Abstract: In the current era of deep learning and especially generative models, there is significant investment in training very large generative models.
By San Buchanan, Druv Pai, Peng Wang, Yi Ma
FlowGenesis is a framework that lets neural networks self‑organize their architecture during training by adding or removing neurons based on a local statistic called Representation Flux. Neurons with high Flux are split to preserve function, while low‑Flux, low‑activity neurons are pruned. Experiments on image and text classification show that networks starting from different sizes converge to similar neuron counts and performance, and the method also improves continual learning by reducing parameters for new tasks without sacrificing accuracy.
By Maksim A Kazanskii
The paper discusses tensorizing neural networks by reshaping dense weight matrices into higher-order tensors and approximating them with low-rank tensor network decompositions. This approach offers promising model compression and introduces bond indices that create new latent spaces, potentially enhancing interpretability. Despite encouraging empirical results, tensorized neural networks remain underused, and the authors call for more research to address practical scaling and adoption challenges.
By Safa Hamreras, Sukhbinder Singh, Rom\'an Or\'us
arXiv:2610.34015v1 Announce Type: cross
Abstract: Most analyses of neural representations focus on trained endpoints, leaving the process by which internal structure forms comparatively unmeasured. W...
By Sanjukta Krishnagopal
arXiv:2605. 22472v2 Announce Type: replace Abstract: Winner-take-all (WTA) networks constitute a central circuit motif in cortical networks of the brain.
By Julian Gutheil (Graz University of Technology), Simon Hitzginger (Graz University of Technology), Robert Legenstein (Graz University of Technology)
Recent advances in Diffusion Transformers (DiTs) have enabled remarkable progress in visual synthesis, benefiting from their superior scalability. To facilitate DiTs' capability of capturing meaningful internal representations, recent works such as REPA incorporate external pretrained encoders for representation alignment.