GAPCNN with HyPar: Global Average Pooling convolutional neural network with novel NNLU activation function and HYBRID...

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep learning, particularly Convolutional Neural Networks (CNNs), is increasingly vital for image recognition, classification, and regression.
• Training deep neural networks is computationally intensive, often requiring days or weeks, hindering real-time applications.
• Existing parallelism strategies like data and model parallelism have limitations, necessitating improved computational speedup for CNNs.

Purpose of the Study:

• To accelerate the training of Convolutional Neural Networks (CNNs) for practical, real-time applications.
• To develop a hybrid parallelism approach combining data and model parallelism to enhance computational speed without sacrificing accuracy.
• To introduce a novel activation function and optimize CNN architecture for improved performance and accuracy.

Main Methods:

• Implemented a hybrid parallelism strategy integrating both data and model parallelism for CNN training.
• Proposed a novel activation function, Normalized Non-linear Activation Unit (NNLU), designed to be non-saturating and mitigate vanishing gradients.
• Replaced fully connected layers with Global Average Pooling (GAP) layers to improve model accuracy and computational efficiency.

Main Results:

• Achieved a significant computational speedup of 3.62X with only a 1.5% drop in accuracy.
• The novel NNLU activation unit demonstrated effectiveness in preventing model over-fitting and avoiding the vanishing gradient problem.
• On a biomedical image dataset, the proposed model attained 98.89% accuracy in classifying glioma, meningioma, and pituitary tumors, with a training time of just 1 second.

Conclusions:

• The hybrid parallelism approach effectively accelerates CNN training while maintaining high accuracy.
• The NNLU activation function and GAP layers contribute to enhanced model performance and robustness.
• The proposed CNN model demonstrates superior classification accuracy and computational speed compared to state-of-the-art methods for medical image analysis.