AMITE: A Novel Polynomial Expansion for Analyzing Neural Network Nonlinearities

Area of Science:

• * Artificial Intelligence
• * Computational Mathematics

Background:

• * Polynomial expansions are crucial for analyzing neural network nonlinearities, aiding verification, explainability, and security.
• * Existing methods like Taylor and Chebyshev expansions have limitations, lacking a consistent approach to combine desirable properties.
• * There's a need for a unified expansion method offering properties such as exact error formulas and adjustable domains.

Purpose of the Study:

• * To introduce a novel expansion technique, analytically modified integral transform expansion (AMITE), for neural network analysis.
• * To address the limitations of existing methods by providing a consistent expansion with multiple desired properties.
• * To demonstrate the applicability and effectiveness of AMITE in analyzing neural network nonlinearities.

Main Methods:

• * Development of AMITE, an integral transform expansion modified with convergence criteria.
• * Application of AMITE to hyperbolic tangent and rectified linear unit activation functions.
• * Comparison of AMITE with classical Chebyshev, Taylor, and numerical expansion methods.

Main Results:

• * AMITE is the first method to consistently provide six previously mutually exclusive expansion properties.
• * AMITE yields exact formulas for coefficients and exact expansion errors.
• * Demonstrated effectiveness in extracting polynomial forms from black-box MLPs and range bounding FFNNs.

Conclusions:

• * AMITE offers a significant advancement in analyzing and approximating neural network nonlinearities.
• * The method enhances theoretical analysis and systematic testing of neural networks.
• * AMITE opens new avenues for research in AI verification, explainability, and security.