Langevin machine: a neural network based on stochastically justifiable sigmoidal function

In neural networks the activation process controls the output as a nonlinear function of the input; and, this output remains bounded between limits as decided by a logistic function known as the sigmoid (S-shaped). Presently, by applying the considerations of Maxwell-Boltzmann statistics, the Langevin function is shown as the appropriate and justifiable sigmoid (instead of the conventional hyperbolic tangent function) to depict the bipolar nonlinear logic-operation enunciated by the collective stochastical response of artificial neurons under activation. That is, the graded response of a large network of 'neurons' such as Hopfield's can be stochastically justified via the proposed model. The model is consistent with the established link between the Hopfield model and the statistical mechanics. The Langevin function (in lieu of conventional hyperbolic tangent and/or exponential sigmoids) in determining nonlinear decision boundaries, in characterizing the neural networks by the Langevin machine versus the Boltzmann machine, in sharpening and annealing schedules and in the optimization of nonlinear detector performance are discussed.