The article "Diversity of EML-type operators" discusses the EML operator, which can evaluate standard explicit purely transcendental elementary functions, and notes that while most research has focused on the binary EML, many similar variants exist. It enumerates and classifies these variants, clarifies common misconceptions, and proposes a M"obius layer that replaces matrix operations with rational functions. The paper also showcases the activation function eml(x,1/x), enabling separate recovery of exp(x) and ln(x) and thus evaluation of all elementary functions within a rational neural‑network generalization.
The recently introduced EML (Exp-Minus-Log) function acts as continuous analogue of NAND gates, providing a compositional building block capable of representing elementary functions. In this work, we study the expressive power of tree-structured compositions of EML functions.
arXiv:2605. 01702v2 Announce Type: replace Abstract: Theoretical studies show that for any differentiable function on a compact domain, there exists a neural network that approximates both the function values and gradients.
arXiv:2607. 09235v1 Announce Type: cross Abstract: The current state of the art in AI/ML rests on deep neural architectures, which, in general, suffer from a lack of interpretability.
NeuralCert presents a framework that learns high‑dimensional variational trial functions in a compact separable form, then spectrally diagnoses, prunes, and exactly certifies them via multimodular evaluation. The method is fully explicit and independently verifiable, and can run on a standard personal computer. Applied to three extremal problems, it demonstrates that neural optimization can discover better constructions, reveal empirical invariants useful for proofs, and expose optimization barriers that inspire new analytic or numerical approaches.
arXiv:2602. 12390v2 Announce Type: replace Abstract: We study neural networks with trainable low-degree rational activation functions and show that they are more expressive and parameter-efficient than modern piecewise-linear and smooth activations such as ELU, LeakyReLU, LogSigmoid, PReLU, ReLU, SELU, CELU, Sigmoid, SiLU, Mish, Softplus, Tanh, Softmin, Softmax, and LogSoftmax.
The paper presents a complete characterization of when two deep ReLU networks realize the same function, showing that this occurs iff one can be transformed into the other using a set of axioms from many‑valued logic. It introduces a symbolic calculus that maps networks to substitution graphs, proves a completeness theorem linking equivalent formulas, and provides an algorithm to reconstruct networks from these graphs. The framework yields a new compositional normal form for MV logic that preserves the algebraic structure of deep ReLU networks.
arXiv:2507. 11688v4 Announce Type: replace Abstract: Contemporary large models often exhibit behaviors suggesting the presence of low-level primitives that compose into modules with richer functionality, but these fundamental building blocks remain poorly understood.
arXiv:2608.29530v1 Announce Type: cross Abstract: Modern systems in artificial intelligence (AI) somehow excel in domains for which they seem poorly suited. Intelligence has traditionally been modele...
SMILE (Sine, Multiplication, Identity, Logarithm, Exponential) is a hybrid framework that merges continuous gradient-based optimization with discrete symbolic recovery for symbolic regression. It operates in three stages: structural analysis to uncover the compositional hierarchy of the target expression, continuous optimization to learn parameters of a network using interpretable activations, and symbolic recovery via structured pruning, coefficient optimization, and rounding to produce a compact expression with exact symbolic constants. Evaluated on SRBench, SMILE achieves the highest symbolic solution rate under high noise, remains on the Pareto front of accuracy versus complexity, and recovers simpler expressions much faster than competing methods.
arXiv:2606. 12318v1 Announce Type: cross Abstract: Neural operators approximate mappings between function spaces, but often generalize poorly to other operators and usually require fine-tuning or retraining.
arXiv:2511. 14953v2 Announce Type: replace-cross Abstract: Discrete structures are currently second-class in differentiable programming.
arXiv:2607. 11289v1 Announce Type: cross Abstract: Backpropagation is the computational engine of deep learning, yet its mathematical structure is typically treated as a procedural traversal of computational graphs.