A novel scaled-gamma-tanh (SGT) activation function in 3D CNN applied for MRI classification

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Activation functions are crucial for neural network performance, introducing non-linearity and enabling complex pattern learning.
• Existing activation functions like ReLU and tanh have limitations, including potential vanishing/exploding gradient issues.
• Feature redundancy reduction and task-specific goal learning are key challenges in deep neural networks.

Purpose of the Study:

• To propose a novel activation function, Scaled Gamma Tanh (SGT), combining scaled gamma correction and hyperbolic tangent.
• To analyze the behavior and effectiveness of SGT against established activation functions.
• To evaluate SGT's performance in a 3D Convolutional Neural Network (CNN) for magnetic resonance imaging (MRI) classification.

Main Methods:

• The SGT activation function involves a two-step process: scaled gamma correction followed by hyperbolic tangent squashing.
• Learnable channel-based parameters ([Formula: see text], [Formula: see text]) and user-defined constants (a, b) are incorporated into SGT.
• SGT was implemented in a 3D CNN for MRI classification, with analyses including input/output histograms, weight/bias plots, and t-SNE projections.

Main Results:

• The proposed SGT activation function demonstrated superior performance in MRI classification compared to ReLU and tanh.
• SGT outperformed standard activations across all evaluated metrics: final validation accuracy, final validation loss, test accuracy, Cohen's kappa score, and Precision.
• Analysis of activation layer inputs/outputs and network weights supported the efficacy of the SGT function.

Conclusions:

• The Scaled Gamma Tanh (SGT) activation function offers a promising alternative to existing methods for deep neural networks.
• SGT effectively addresses challenges related to non-linearity and gradient stability in deep learning models.
• The superior performance of SGT in MRI classification highlights its potential for medical image analysis and other complex tasks.