Learning Spatiotemporal Features for Esophageal Abnormality Detection From Endoscopic Videos

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Gastroenterology

Background:

• Esophageal cancer has a high mortality rate, emphasizing the need for early detection of abnormalities.
• Current deep learning methods primarily focus on still endoscopic images, lacking video-based detection capabilities.
• Detecting multiple esophageal abnormality types from challenging video frames remains an unmet challenge.

Purpose of the Study:

• To develop an efficient deep learning method for automatic detection of diverse esophageal abnormalities from endoscopic videos.
• To address limitations in existing methods by enabling video analysis, handling challenging frames, and identifying multiple abnormality types.

Main Methods:

• Proposed a novel 3D Sequential DenseConvLstm network to extract spatiotemporal features from endoscopic videos.
• Integrated 3D Convolutional Neural Network (3DCNN) and Convolutional LSTM (ConvLSTM) for learning short and long-term features.
• Employed a region proposal network, ROI pooling, and a Frame Search Conditional Random Field (FS-CRF) for detection and performance enhancement.

Main Results:

• Achieved high performance on an esophageal abnormality dataset with 93.7% recall, 92.7% precision, and 93.2% F-measure.
• Demonstrated robustness by achieving 81.18% recall, 96.45% precision, and 88.16% F-measure for polyp detection on a public colonoscopy dataset, outperforming state-of-the-art.
• The method shows adaptability for various gastrointestinal endoscopic video applications.

Conclusions:

• The proposed 3D Sequential DenseConvLstm network effectively detects esophageal abnormalities in endoscopic videos.
• The FS-CRF post-processing method enhances detection accuracy by refining results across frames.
• The model's strong performance and adaptability suggest significant potential for improving endoscopic diagnostics in gastroenterology.