A Novel 3D Convolutional Neural Network-Based Deep Learning Model for Spatiotemporal Feature Mapping for Video...

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Accurate medical video analysis is challenging due to the need for effective spatiotemporal feature extraction.
• Existing deep learning models often focus on static images, neglecting temporal dynamics crucial for video data.
• Gastrointestinal (GI) endoscopic video classification presents a specific challenge in capturing subtle visual cues over time.

Purpose of the Study:

• To propose a novel deep learning framework for spatiotemporal feature extraction from medical video sequences.
• To develop and evaluate a 3D convolutional neural network (CNN) for classifying upper and lower GI endoscopic videos.
• To introduce and integrate a novel residual with parallel attention (RPA) block for enhanced feature learning.

Main Methods:

• A 3D CNN architecture was developed for GI endoscopic video classification using the hyperKvasir dataset.
• Data imbalance was addressed by selecting matched pairs of videos across multiple experimental runs.
• A (2+1)D convolution was employed to reduce computational complexity, and a novel RPA block combining P-scSE3D with residual connections was implemented.

Main Results:

• The proposed model achieved high performance metrics, including an average accuracy of 0.933, precision of 0.932, recall of 0.944, and F1-score of 0.935.
• Integration of the 3D parallel spatial and channel squeeze-and-excitation (P-scSE3D) module resulted in a 7% increase in the F1-score.
• The model demonstrated robust performance in classifying upper and lower GI endoscopic videos.

Conclusions:

• The developed deep learning framework effectively extracts spatiotemporal features for medical video analysis.
• The novel RPA block and (2+1)D convolution contribute to improved accuracy and computational efficiency.
• This work paves the way for advanced GI endoscopic video analysis and future clinical applications.