Multi-Modal Graph Neural Networks for Colposcopy Data Classification and Visualization.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Oncology

Background:

• Cervical lesion classification is critical for early cervical cancer detection.
• Current deep learning models often use single-modal data or require extensive manual annotations.
• A novel Graph Neural Network (GNN) framework is proposed to integrate multi-modal data for enhanced classification.

Purpose of the Study:

• To develop and evaluate a GNN-based framework for cervical lesion classification.
• To improve the accuracy and efficiency of cervical cancer detection using multi-modal data.
• To explore the potential of graph-based multi-modal learning in clinical oncology.

Main Methods:

• A fully connected graph-based architecture using GCNConv layers and global mean pooling was developed.
• Model optimization was performed using grid search, with performance evaluated via five-fold cross-validation.
• The framework integrates colposcopy images, segmentation masks, and graph representations.

Main Results:

• The GNN model achieved a macro-average F1-score of 89.4% and validation accuracy of 92.1% before fine-tuning.
• Performance improved to 94.56% (F1-score) and 98.98% (accuracy) after fine-tuning.
• LIME-based visualizations confirmed the model's focus on discriminative lesion regions.

Conclusions:

• Graph-based multi-modal learning shows significant potential for cervical lesion analysis.
• The developed framework demonstrates promise for clinical application in cervical cancer screening.
• Collaboration with clinical institutions like the MNJ Institute of Oncology is valuable for translational research.