Self-learning activation functions to increase accuracy of privacy-preserving Convolutional Neural Networks with...

Area of Science:

• Computer Science
• Cryptography
• Machine Learning

Background:

• Cloud computing adoption necessitates robust privacy-preserving techniques for data processing.
• Existing privacy-preserving methods for Convolutional Neural Networks with Homomorphic Encryption (CNN-HE) face challenges in accuracy and performance.
• Activation functions are critical components in CNNs, influencing their learning capabilities.

Purpose of the Study:

• To propose and evaluate Self-Learning Activation Functions (SLAFs) for enhancing the accuracy and performance of privacy-preserving CNN-HE models.
• To theoretically prove the feasibility of SLAFs in approximating continuous activation functions.
• To introduce two novel CNN-HE models incorporating SLAFs: CNN-HE-SLAF and CNN-HE-SLAF-R.

Main Methods:

• Developed SLAFs as trainable polynomials with coefficients updated during training, independent of synaptic weights.
• Proposed two CNN-HE architectures: CNN-HE-SLAF (all activations replaced by SLAFs) and CNN-HE-SLAF-R (SLAFs adapted after initial training).
• Theoretically demonstrated the approximation capability of SLAFs for any continuous activation function based on polynomial degree.

Main Results:

• Achieved 99.38% accuracy on the MNIST dataset, comparable to non-homomorphic CNNs using ReLU.
• The CNN-HE-SLAF-R model demonstrated improved accuracy (99.21%) and a significant performance increase (6.26 times faster) compared to state-of-the-art CNN-HE CryptoNets.
• SLAFs enable task-specific and CNN-specific feature learning within the homomorphic encryption framework.

Conclusions:

• Self-Learning Activation Functions offer a viable method to enhance both accuracy and performance of privacy-preserving CNN-HE.
• SLAFs provide a flexible and efficient alternative to traditional activation functions in homomorphic encryption settings.
• The proposed CNN-HE-SLAF models represent a significant advancement in secure and efficient deep learning on encrypted data.