Related Experiment Video
Updated: Sep 4, 2025

Classification of Neural Stem Cell Activation State In Vitro using Autofluorescence
Published on: April 12, 2024
A Formal Characterization of Activation Functions in Deep Neural Networks
Abstract:
In this article, a mathematical formulation for describing and designing activation functions in deep neural networks is provided. The methodology is based on a precise characterization of the desired activation functions that satisfy particular criteria, including circumventing vanishing or exploding gradients during training. The problem of finding desired activation functions is formulated as an infinite-dimensional optimization problem, which is later relaxed to solving a partial differential equation. Furthermore, bounds that guarantee the optimality of the designed activation function are provided. Relevant examples with some state-of-the-art activation functions are provided to illustrate the methodology.
Related Concept Videos
Activation and Inactivation of G Proteins
Neural Regulation
Activation Energy
Neural Circuits
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Propagation of Action Potentials
Neurons (nerve cells) have a resting membrane potential, with a slightly negative charge inside compared to outside. This is maintained by ion channels, such as sodium (Na+) and potassium (K+) channels, which control the flow of ions. When a stimulus, like a touch or a signal from another neuron, triggers the neuron, sodium channels open, allowing sodium ions to...
The Role of Ion Channels in Neuronal Computation
Sometimes a single EPSP is strong enough to induce an action potential in the postsynaptic neuron. However, multiple presynaptic inputs must often create EPSPs around the same time for the postsynaptic neuron to be sufficiently depolarized to fire an action potential....

