A Novel Image Classification Method Based on Residual Network, Inception, and Proposed Activation Function.

Area of Science:

• Computer Science
• Artificial Intelligence
• Deep Learning

Background:

• ResNet architectures rely on skip connections and ReLU, facing challenges with inconsistent layer dimensions and vanishing gradients.
• Dimension mismatches in ResNet require zero-padding or projection, increasing complexity, parameters, and computational costs.
• The ReLU activation function can lead to vanishing gradients, hindering network training.

Purpose of the Study:

• To enhance ResNet performance by addressing limitations in deeper layers.
• To reduce parameter count and computational overhead in deep learning models.
• To improve model accuracy and convergence speed through a novel activation function.

Main Methods:

• Modified inception blocks were integrated into deeper ResNet layers.
• The ReLU activation function was replaced with a non-monotonic activation function (NMAF).
• Symmetric factorization and 1x1 convolutions were employed to reduce parameter numbers.

Main Results:

• Parameter count was reduced by approximately 6 million.
• Training runtime decreased by 30 seconds per epoch.
• Accuracy improvements of 5-21% were observed across noisy and non-noisy datasets.
• NMAF addressed ReLU's deactivation problem by activating negative values, enhancing convergence.

Conclusions:

• The proposed modified ResNet with inception blocks and NMAF offers a more efficient and accurate deep learning architecture.
• NMAF effectively mitigates vanishing gradients and improves training dynamics compared to ReLU.
• The architectural modifications significantly reduce computational costs and parameter count without sacrificing performance.