Intelligent Malaysian Sign Language Translation System Using Convolutional-Based Attention Module with Residual...

Area of Science:

• Computer Vision
• Artificial Intelligence
• Human-Computer Interaction

Background:

• The deaf-mute population faces communication barriers due to the lack of understanding of sign language.
• Developing effective sign language recognition systems is crucial for bridging this communication gap.

Purpose of the Study:

• To implement and evaluate a Convolutional Neural Network (CNN) with a Convolutional Block Attention Module (CBAM) for recognizing Malaysian Sign Language (MSL) from images.
• To compare the efficiency of two different implementation methods ('Within Blocks' and 'Before Classifier') of the CBAM-2DResNet model for MSL recognition.

Main Methods:

• Utilized a Convolutional Neural Network (CNN) architecture integrated with a Convolutional Block Attention Module (CBAM).
• Conducted experiments using CBAM-2DResNet (2-Dimensional Residual Network) with two distinct configurations: 'Within Blocks' and 'Before Classifier'.
• Evaluated model performance using metrics including accuracy, loss, precision, recall, F1-score, confusion matrix, and training time.

Main Results:

• CBAM-ResNet models demonstrated strong performance in MSL sign recognition, achieving accuracy rates exceeding 90%.
• The 'Before Classifier' configuration of CBAM-ResNet proved more efficient than the 'Within Blocks' configuration.
• The best-performing CBAM-2DResNet model was selected for developing a real-time sign language translation system.

Conclusions:

• The CBAM-ResNet models are highly effective for Malaysian Sign Language recognition.
• The 'Before Classifier' approach offers superior efficiency for MSL recognition tasks.
• The developed system has the potential to significantly enhance communication for the deaf-mute community.