Neural Symbolic Regression (NSR) uses neural networks as functional preconditioners to learn smooth, noise‑robust approximations of target functions in an interaction‑aware nonlinear feature space. A subsequent LASSO step extracts sparse, interpretable closed‑form expressions, while distributed hyperparameter optimization with Ray Tune and ASHA scheduling improves predictive accuracy and symbolic fidelity. Experiments on the Nguyen benchmark demonstrate that NSR outperforms SINDy and untuned neural baselines in RMSE, noise robustness, and out‑of‑distribution generalization, with ablation studies highlighting the importance of feature interactions, neural depth, and tuning strategies.
By Ravi Kumar U, Sumitra S
arXiv:2605. 23272v2 Announce Type: replace-cross Abstract: Symbolic Regression (SR) plays a central role in scientific knowledge discovery by distilling mathematical equations from observational data.
By Boxiao Wang, Kai Li, Zhiwei Chen, Yang Huang, Runxiang Wang, Ziwen Zhang, Yifan Zhang, Jian Cheng
arXiv:2608. 02628v1 Announce Type: cross Abstract: Symbolic regression (SR) is the task of discovering underlying patterns from data and representing them using mathematical expressions.
By Yusong Deng, Yanjie Li, Weijun Li
SMILE (Sine, Multiplication, Identity, Logarithm, Exponential) is a hybrid framework that merges continuous gradient-based optimization with discrete symbolic recovery for symbolic regression. It operates in three stages: structural analysis to uncover the compositional hierarchy of the target expression, continuous optimization to learn parameters of a network using interpretable activations, and symbolic recovery via structured pruning, coefficient optimization, and rounding to produce a compact expression with exact symbolic constants. Evaluated on SRBench, SMILE achieves the highest symbolic solution rate under high noise, remains on the Pareto front of accuracy versus complexity, and recovers simpler expressions much faster than competing methods.
By Mansooreh Montazerin, Antonio Ortega, Ajitesh Srivastava
arXiv:2511. 04124v3 Announce Type: replace Abstract: Symbolic regression (SR) models complex systems by discovering mathematical expressions that capture underlying relationships in observed data.
By Giorgio Morales, John W. Sheppard
arXiv:2606. 07915v1 Announce Type: new Abstract: Neural symbolic regression models improve inference efficiency by shifting structural search to pretraining, but their one-pass autoregressive decoding is prone to error accumulation, which may lead to generating structurally incorrect expressions, especially in complex expression generation scenarios.
By Da Li, Xinxin Li, Xingyu Cui, Jin Xu, Juan Zhang, Junping Yin
arXiv:2607. 09967v1 Announce Type: cross Abstract: Many neural networks operations have a multiplicative nature rather than additive: halving or doubling a norm are analogous relatively but require unequal optimization distances when taking linear steps.
By Ethan Smith
arXiv:2610.01519v1 Announce Type: cross
Abstract: Neuro-Symbolic (NeSy) predictors incorporate prior knowledge into the prediction process of neural networks, ensuring that outputs satisfy specified...
By Samuele Bortolotti, Weixin Chen, Han Zhao, Andrea Passerini, Stefano Teso, Antonio Vergari
arXiv:2405. 04376v4 Announce Type: replace Abstract: Hyperparameter tuning, particularly the selection of an appropriate learning rate in adaptive gradient training methods, remains a challenge.
By Yijiang Pang, Shuyang Yu, Bao Hoang, Jiayu Zhou
The article surveys recent neural‑network optimizers, noting that the field has moved beyond simple Adam variants to encompass matrix‑ and layer‑level designs, time‑policy horizons, and state representations that survive sharding and low‑precision computation. It categorizes optimizers along four axes—temporal estimation, update geometry, horizon management, and representation & systems—highlighting methods such as Muon, Shampoo, SOAP, and quantized states. The survey concludes that while matrix‑aware methods are a genuine advance, no single optimizer universally replaces AdamW, and performance depends on model scale, data‑to‑parameter ratio, batch size, schedule, partitioning, tuning budget, and target metric.
By Ruoran Xu
arXiv:2606. 25971v1 Announce Type: new Abstract: Modern neural network training relies on optimizers such as Adam and Muon which act on each weight matrix as a single object.
By Alexander H\"agele, Alejandro Hern\'andez-Cano, Atli Kosson, Martin Jaggi
arXiv:2607. 20548v1 Announce Type: cross Abstract: Higher-order optimizers such as Muon and SOAP offer faster convergence than AdamW, but their computational cost and numerical stability challenges have limited adoption at scale.
By Mikail Khona, Aditya Vavre, Boxiang Wang, Deyu Fu, Hao Wu, Mike Chrzanowski, Bryan Catanzaro, Dheevatsa Mudigere, Jeff Pool, Michael Lightstone, Mohammad Shoeybi, Mostofa Patwary, Nima Tajbakhsh, Tijmen Blankevoort