A High-Accuracy Probabilistic-Based Sigmoid Approximator Incorporating Memory-Saving and Time-Efficient Strategies.

Area of Science:

• Computer Science
• Artificial Intelligence
• Hardware Engineering

Background:

• The sigmoid function is crucial for neural networks, especially on edge devices.
• Previous probabilistic approximations using Gaussian cumulative functions faced challenges with memory usage, speed, and accuracy across all inputs.

Purpose of the Study:

• To develop a hardware-friendly and highly accurate probabilistic sigmoid approximator.
• To overcome the limitations of existing methods, such as high RAM requirements and time-consuming processes.

Main Methods:

• Establishing the equivalence between sigmoid function output and the probability of a logistic random variable.
• Implementing an indirect random variable quantizing strategy to minimize memory and precision loss.
• Optimizing latency and developing a resource-efficient digital circuit implementation.

Main Results:

• The proposed scheme significantly reduces memory usage and precision loss.
• The developed sigmoid approximator demonstrates optimized latency and resource efficiency.
• An upper bound on the absolute error was derived, confirming approximation accuracy.

Conclusions:

• The novel probabilistic sigmoid approximator offers superior performance in accuracy and resource cost compared to existing methods.
• This approach provides a hardware-friendly and efficient solution for sigmoid approximation in neural networks, particularly for edge computing applications.