Deep Graph-Based Multimodal Feature Embedding for Endomicroscopy Image Retrieval.

Area of Science:

• Medical Image Analysis
• Computer-Aided Diagnosis
• Representation Learning

Background:

• Learning discriminative features for medical image analysis is challenging due to limited data and lack of labels.
• Probe-based confocal laser endomicroscopy (pCLE) generates data requiring effective feature extraction for classification.

Purpose of the Study:

• To propose a deep graph-based multimodal feature embedding (DGMFE) framework for medical image retrieval.
• To apply DGMFE for breast tissue classification using pCLE data.
• To learn discriminative features for enhanced computer-aided diagnosis.

Main Methods:

• Constructed a multimodality graph model based on visual similarity between pCLE and histology images.
• Extracted similar pCLE-histology pairs using cyclic path graph traversal and dissimilar pairs via geodesic distance.
• Discovered latent feature space using deep Siamese neural networks to reconstruct image similarity.

Main Results:

• The DGMFE framework demonstrated superior performance in medical image retrieval compared to previous methods.
• Achieved a top-1 accuracy of 0.739 in an eight-class retrieval task, a 10% improvement over state-of-the-art.
• Evaluation on a clinical database of 700 pCLE mosaics validated the method's effectiveness.

Conclusions:

• The proposed DGMFE framework effectively learns discriminative features for medical image analysis.
• DGMFE significantly improves accuracy in breast tissue classification and medical image retrieval tasks.
• This approach offers a promising advancement for computer-aided diagnosis using multimodal medical imaging data.