The paper investigates how temperature affects analog deep neural network (DNN) inference, focusing on both stochastic and systematic non‑idealities in analog hardware. Experiments show that temperature‑induced performance loss is mainly driven by systematic errors rather than random noise. The study evaluates various mitigation techniques, finding that noise‑aware training and temperature‑aware calibration—especially hardware‑in‑the‑loop training—best preserve inference accuracy across different thermal conditions.
By Niklas Summ, Xiao Wang, Hendrik Borras, Bernhard Klein, Holger Fr\"oning
The paper introduces a communication‑efficient method for adapting large language models on decentralized GPU meshes. It proposes an asynchronous two‑circuit system that uses fast compressed training with activation masking for pipeline‑parallel transfer and compressed data‑parallel synchronization, while a slower anchor circuit performs occasional unmasked passes. A spectral correction optimizer then denoises the masked gradients using these anchor priors, enabling high compression rates and achieving up to 40× throughput gains over internet‑grade connections while matching dense uncompressed performance.
By Sameera Ramasinghe, Shamane Siriwardhana, Thalaiyasingam Ajanthan, Hadi Mohaghegh Dolatabadi, Chamin P Hewa Koneputugodage, Gil Avraham, Violetta Shevchenko, James Snewin, Karol Pajak, Harry Xi, Alexander Long
arXiv:2505. 20137v5 Announce Type: replace-cross Abstract: Predictive Coding (PC) offers a brain-inspired alternative to backpropagation for neural network training, described as a physical system minimizing its internal energy.
By C\'edric Goemaere, Gaspard Oliviers, Rafal Bogacz, Thomas Demeester
arXiv:2605. 11855v2 Announce Type: replace-cross Abstract: Sequence learning is dominated by Transformers and parallelizable recurrent neural networks (RNNs) such as state-space models, yet learning long-term dependencies remains challenging, and state-of-the-art designs trade power consumption for performance.
By Julien Brandoit, Arthur Fyon, Damien Ernst, Guillaume Drion
arXiv:2608. 01997v1 Announce Type: new Abstract: Single-optimizer training is a poor fit for the distinct phases of deep network optimization: adaptive methods handle noisy early gradients well but overshoot flat minima, while SGD with momentum generalizes better in the late phase but converges slowly early on.
By Alok Kumar Pandey, Umang Chaturvedi, Aatish Rana, Gopi Krishna Nedanuri
arXiv:2609.36584v1 Announce Type: new
Abstract: Analog in-memory computing (AIMC) offers an alternative for model training by executing matrix operations directly where weights are stored. However, s...
By Zhaoxian Wu, Tayfun Gokmen, Omobayode Fagbohungbe, T. Patrick Xiao, Tianyi Chen
arXiv:2606. 23129v2 Announce Type: replace-cross Abstract: Implicit Neural Representations (INRs) have been proven successful in encoding continuous signals through coordinate-based networks, yet facing a spectral dilemma: periodic activations capture fine details but act as all-pass filters that memorise noise, while spatially compact activations regularise effectively but suffer from low-frequency bias.
By Alex Costanzino, Pierluigi Zama Ramirez, Giuseppe Lisanti, Luigi Di Stefano
arXiv:2606. 03584v1 Announce Type: new Abstract: Equilibrium Propagation (EP) is a physics-based training framework that has primarily been employed in energy-based models, including continuous Hopfield networks, nonlinear resistive networks and coupled phase oscillators.
By Tugdual Kerjan, Rasmus H{\o}ier, Benjamin Scellier
arXiv:2606. 00888v1 Announce Type: cross Abstract: Dynamic Sparse Training (DST) offers a promising paradigm for improving the training and inference efficiency of deep neural networks; however, we find that in large language model training, DST can suffer from optimization instability, manifested as loss spikes after topology updates.
By Qiao Xiao, Boqian Wu, Patrik Okanovic, Tomasz Sternal, Maurice van Keulen, Elena Mocanu, Mykola Pechenizkiy, Decebal Constantin Mocanu, Torsten Hoefler
arXiv:2603. 10562v2 Announce Type: replace-cross Abstract: Monotone operator equilibrium networks are implicit-layer models whose output is the unique equilibrium of a monotone operator, guaranteeing existence, uniqueness, and convergence.
By James Li, Philip H. W. Leong, Thomas Chaffey
arXiv:2606. 15443v1 Announce Type: cross Abstract: Physical learning methods train physical networks to perform computational tasks using only local update rules, exploiting the physics of the system to handle the global transfer of information.
By Joshua A. McGinnis, Xinbo Li, Yoichiro Mori
arXiv:2607. 00286v1 Announce Type: cross Abstract: Oscillatory neural networks (ONNs) have emerged as a promising neuromorphic architecture, leveraging coupled dynamical systems to perform computation and represent information through phase relationships.
By Riley Acker, Aman Desai, Garrett Kenyon, Frank Barrows