Classification of endoscopic image and video frames using distance metric-based learning with interpolated latent...

Area of Science:

• Medical Imaging
• Artificial Intelligence in Medicine
• Gastroenterology

Background:

• Conventional Endoscopy (CE) and Wireless Capsule Endoscopy (WCE) are crucial for diagnosing gastrointestinal (GI) disorders.
• Accurate anatomical localization in endoscopic imaging is vital for effective treatment planning and reducing redundant procedures.
• Limited research exists on classifying endoscopic images due to challenges in acquiring annotated data.

Purpose of the Study:

• To develop a few-shot learning method for localizing and classifying anatomical locations in CE and WCE images.
• To create an automated pipeline for endoscopy video sequence localization trainable with minimal annotated samples.
• To improve the accuracy and efficiency of endoscopic image analysis.

Main Methods:

• A few-shot learning approach combining distance metric learning, transfer learning, and manifold mixup.
• Training models on a dataset from 10 GI tract anatomical positions using very few annotated frames (78 CE, 27 WCE).
• Evaluating the method's performance against subjective assessments by gastroenterologists and existing techniques.

Main Results:

• The proposed method achieved higher accuracy and F1-score compared to subjective evaluations.
• Demonstrated improved performance with lower cross-entropy loss against several existing methods.
• Successfully localized and classified a large number of video frames (25,700 CE, 1825 WCE) with limited training data.

Conclusions:

• The few-shot learning method shows significant potential for accurate endoscopy image classification and localization.
• The approach effectively addresses the challenge of limited annotated data in endoscopic imaging.
• This automated system can assist clinicians in diagnosing GI disorders and optimizing treatment strategies.