A smart atlas for endomicroscopy using automated video retrieval

Area of Science:

• Medical Imaging
• Computer Vision
• Oncology

Background:

• Early epithelial cancer diagnosis is challenging, especially with in vivo endomicroscopy.
• Current methods may lack sufficient field-of-view for robust diagnosis from single images.

Purpose of the Study:

• To develop a content-based video retrieval method for early epithelial cancer diagnosis using expert-annotated endomicroscopy data.
• To improve diagnostic accuracy by integrating image and video analysis techniques.

Main Methods:

• Adapted Bag-of-Visual-Words for endomicroscopic images with local dense multi-scale descriptions for invariance.
• Developed a video-mosaicing technique for large field-of-view images and a geometrical approach for feature relationship analysis.
• Implemented efficient video retrieval using coarse registration to avoid time-consuming video mosaicking.

Main Results:

• Achieved a binary classification accuracy of 94.2%, nearing clinical expectations.
• Demonstrated statistically significant outperformance compared to several state-of-the-art methods in both binary and multi-class classification.
• The method effectively handles translations, rotations, and affine transformations in endoscopic images.

Conclusions:

• The proposed content-based video retrieval method significantly enhances early epithelial cancer diagnosis from in vivo endomicroscopy.
• This approach offers a promising tool for improving diagnostic accuracy and clinical decision-making in oncology.
• The integration of image retrieval and video analysis techniques provides a robust solution for endoscopic video analysis.