The paper reports on the updated AlgoPerf V1 benchmark, evaluating new training algorithms on wall‑clock time‑to‑target across nine workloads on fixed hardware. Using a self‑tuning ruleset that requires a single configuration for all workloads, Schedule‑Free AdamW achieved the best overall performance among 13 PyTorch and JAX submissions. The study also examines trade‑offs between step efficiency and runtime, finding that matrix‑preconditioned and two‑loop methods are sample‑efficient but need aggressive computational optimizations, and highlights the high sensitivity of learning rate and momentum across domains, underscoring the need for more robust hyperparameter‑tuning‑resistant methods.
By Priya Kasimbeg (Alice), Ahmed Khaled (Alice), Aahlad Chandrabhatta (Alice), Yifan Wang (Alice), Andy Liu (Alice), Niccol\`o Ajroldi (Alice), David Tweedle (Alice), Xingyu (Alice), Yang, Isaac Patole, Frank Schneider, George E. Dahl
arXiv:2608.30070v1 Announce Type: new
Abstract: Sparse representations are often expected to make models smaller and also reduce inference cost. For Fourier Neural Operators (FNOs), these objectives...
By Abdul Qadir Ibrahim, Martin Burger
arXiv:2609.24021v1 Announce Type: cross
Abstract: Neural operators are data-driven models that learn mappings from inputs that parameterize partial differential equations, such as spatially varying c...
By Daniel Zhengyu Huang, Andrew M. Stuart
Neural operators are data-driven models that learn mappings from inputs that parameterize partial differential equations, such as spatially varying coefficients, initial conditions, forcing terms, bou...
arXiv:2603. 15506v2 Announce Type: replace-cross Abstract: We argue that the current practice of evaluating AI/ML time-series forecasting models, predominantly on benchmarks characterized by strong, persistent periodicities and seasonalities, obscures real progress by overlooking the performance of efficient classical methods.
By Raeid Saqur, Christoph Bergmeir, Blanka Horvath, Daniel Schmidt, Frank Rudzicz, Terry Lyons
arXiv:2608. 00029v1 Announce Type: cross Abstract: The performance of deep learning models at scale relies heavily on how effectively high-level mathematical operations are mapped to underlying physical hardware.
By Adwaid Suresh, Aparna A, Harshini V M, Jona Delcy C A, Killi Uma Maheswara Rao, Ram Charan Golla, Surendra Vendra
arXiv:2506. 05233v2 Announce Type: replace-cross Abstract: Sequence modeling is currently dominated by causal transformer architectures that use softmax self-attention.
By Johannes von Oswald, Nino Scherrer, Seijin Kobayashi, Luca Versari, Songlin Yang, Sarthak Mittal, Maximilian Schlegel, Kaitlin Maile, Yanick Schimpf, Oliver Sieberling, Alexander Meulemans, Rif A. Saurous, Guillaume Lajoie, Charlotte Frenkel, Razvan Pascanu, Blaise Ag\"uera y Arcas, Jo\~ao Sacramento
arXiv:2512. 22702v2 Announce Type: replace Abstract: Deep learning models have grown popular in time series applications.
By Valentina Moretti, Ivan Marisca, Cesare Alippi, Andrea Cini
arXiv:2605.12904v2 Announce Type: replace
Abstract: Tabular foundation models (TFMs) have emerged as a powerful paradigm for in-context learning on structured data, enabling direct prediction on new...
By Yilong Chen, Xueying Ding, Leman Akoglu
The paper surveys continuous‑time (CT) machine learning, a framework for modeling temporal dynamics as continuous processes, especially useful when data are sampled irregularly or over long horizons. It introduces a unified taxonomy that groups major CT methods by their underlying mathematical formulations and shows how different architectural choices—such as vector‑field parameterization, stochasticity, memory mechanisms, and discretization—relate these families. The survey compares training algorithms, optimization strategies, failure modes, computational complexity, and benchmarks, reviews supporting software ecosystems, and outlines open challenges and future research directions.
By Waleed Razzaq, Yun-Sheng Zhao, Yun-Bo Zhao
arXiv:2511. 00044v3 Announce Type: replace Abstract: Physical neural networks (PNNs) are promising candidates for next-generation computing, but existing demonstrations remain several orders of magnitude smaller than modern digital neural networks, whose recent advances have been driven by rapid growth in trainable parameters.
By Kohei Tsuchiyama, Andre Roehm, Takatomo Mihana, Ryoichi Horisaki
arXiv:2606. 17460v1 Announce Type: new Abstract: Neural operators are widely used as surrogate solution maps for partial differential equations (PDEs), but full-size models can be costly to store, deploy, and evaluate in many-query scientific workflows.
By Lennon J. Shikhman