arXiv AI

Per-Node Activation Function Evolution in Indirectly Encoded Substrates: Solvability, Limits, and Emergent Diversity

The paper demonstrates that using a single activation function across all nodes in artificial neural networks imposes hard limits on evolutionary search, particularly for sparse evolved substrates. By evolving per-node activation functions from an 18-function palette, the authors show that oscillatory functions can solve parity problems at all tested scales, while monotonic functions fail beyond the simplest case. The study reveals that the choice of activation functions, beyond topology and weights, critically influences what evolutionary search can achieve, and that heterogeneous assignments discovered via indirect encoding are unlikely to be selected manually.

arXiv AI

2d ago

Multi-Behavioral Evolved Substrates Through Neuromodulation and Activation Selection

The paper investigates whether artificial evolution can replicate biological neuromodulation and diverse neuron types in indirectly encoded substrates. Experiments show that neuromodulation alone cannot overcome a 75% performance ceiling on parity tasks, but combining neuromodulation with per‑task activation function selection allows a single evolving genotype to achieve 100% success across five tasks. This demonstrates that both neuromodulation and evolvable computational primitives are necessary for multi‑behavioral open‑ended evolution.

By Romain Claret, Michael O'Neill, Paul Cotofrei, Kilian Stoffel

arXiv Machine Learning

Sep 16

Bio-Inspired Palette Evolution in Indirectly Encoded Substrates: Timescale Compatibility Shapes Activation Function Discovery

arXiv:2609.17067v1 Announce Type: cross Abstract: Indirectly encoded neural networks can assign different activation functions to individual nodes, but the right functions are rarely known in advance...

By Romain Claret, Michael O'Neill, Paul Cotofrei, Kilian Stoffel

arXiv Machine Learning

Jul 13

Evolutionary Discovery of Developmental Reward Schedules in Deep Reinforcement Learning

arXiv:2606. 20858v2 Announce Type: replace Abstract: The temporal structure of reward composition in reinforcement learning (RL) is typically hand-designed and held fixed throughout training, leaving the progression of motivational priorities largely unexplored.

By Alan Nadelsticher Ruvalcaba

reinforcement-learning

arXiv Machine Learning

Aug 31

Biologically Inspired Mechanisms for Facilitating Grokking in Multilayer Perceptrons

arXiv:2608.28184v1 Announce Type: new Abstract: Grokking is a delayed transition from memorization to generalization that is often accompanied by substantial reorganization of internal representation...

By Florin Leon

llms benchmarks

arXiv AI

Jun 2

Rare Events, Real Signals: Functional Ensembles as Units of Computation in Deep Spiking Networks

arXiv:2606. 00073v1 Announce Type: cross Abstract: We investigate how internal representations emerge across hierarchical processing systems by introducing a neuroscience-inspired framework for analyzing deep spiking neural networks (SNN) through the lens of functional connectivity.

By Aditi Aravind, Konstantinos Ladakis, Mario Alexios Savaglio, Stelios M. Smirnakis, Maria Papadopouli

safety

arXiv Machine Learning

Jul 17

NeuronSoup: Evolving Asynchronous, Shared-Neuron Temporal Graphs without Backpropagation

arXiv:2607. 15217v1 Announce Type: cross Abstract: We present NeuronSoup, a neural computation architecture that replaces synchronous layer-by-layer processing with asynchronous, delay-mediated signal propagation through a pool of shared neurons.

By Subodh Kalia

arXiv Machine Learning

Jul 1

Diffusing Blame: Task-Dependent Credit Assignment in Biologically Plausible Dual-Stream Networks

arXiv:2606. 31700v1 Announce Type: new Abstract: Biological neural circuits obey Dale's principle: each neuron's synapses are uniformly excitatory or inhibitory.

By Yutaro Yamada, Luca Grillotti, Rujikorn Charakorn, Sebastian Risi, David Ha, Robert Tjarko Lange

diffusion reinforcement-learning benchmarks safety

arXiv Machine Learning

Sep 23

Activation-Energy Pruning for Spiking Neural Networks: Unsupervised Personalization via Spike-Count Saliency

arXiv:2609.26167v1 Announce Type: new Abstract: Activation-energy pruning -- removing weights whose product of magnitude and cumulative pre-synaptic spike count falls below a threshold -- was establi...

By Joseph Bingham

efficiency benchmarks

arXiv AI

Jun 16

No One-Size-Fits-All Neurons: Task-based Neurons for Artificial Neural Networks

arXiv:2405. 02369v2 Announce Type: replace-cross Abstract: In the past decade, many successful networks are on novel architectures, which almost exclusively use the same type of neurons.

By Feng-Lei Fan, Meng Wang, Hang-Cheng Dong, Jianwei Ma, Tieyong Zeng

benchmarks safety

arXiv Machine Learning

Jul 30

Emergent Sparsity in Frozen Random CNN Feature Extractors for Deep Reinforcement Learning

arXiv:2607. 26059v1 Announce Type: new Abstract: We report a striking phenomenon: deep reinforcement learning agents trained with frozen, randomly initialized CNN feature extractors spontaneously develop extremely sparse fully-connected representations, without any sparsity-inducing objective.

By Scott M. Norton

agents reinforcement-learning efficiency benchmarks

arXiv AI

Jun 9

SmartMixed: A Two-Phase Training Strategy for Adaptive Activation Function Learning in Neural Networks

arXiv:2510. 22450v3 Announce Type: replace-cross Abstract: The choice of activation function plays a critical role in neural networks, yet most architectures still rely on fixed, uniform activation functions across all neurons.

By Amin Omidvar

benchmarks

arXiv Machine Learning

Jun 18

InTrain: Intrinsic Trainability for Zero-Cost Neural Architecture Search

arXiv:2606. 18676v1 Announce Type: new Abstract: Training-free neural architecture search promises efficient discovery of high-performance networks without costly training.

By Qinqin Zhou, Fuhai Chen, Jipeng Wu, Zhiwei Chen, Zhikai Hu, Weiwei Cai

benchmarks

Related stories