Related Experiment Video
Updated: Sep 9, 2025
04:48
Application of Deep Learning-Based Medical Image Segmentation via Orbital Computed Tomography
Published on: November 30, 2022
2.9K
Explainable self-supervised learning for medical image diagnosis based on DINO V2 model and semantic search
Alaa Hussien1, Abdelkareem Elkhateb1, Mai Saeed1
1Machine Learning and Information Retrieval Department, Faculty of Artificial Intelligence, Kaferelshikh University, Kaferelshikh, 33511, Egypt.
Scientific Reports
|September 1, 2025
Summary
Self-supervised learning (SSL), particularly DINOv2, enhances medical image diagnosis accuracy and interpretability. This approach overcomes data labeling limitations and improves clinical decision-making through efficient semantic search capabilities.
Area of Science:
- Artificial Intelligence in Medicine
- Medical Image Analysis
- Deep Learning
Background:
- Medical imaging is crucial for diagnosis and treatment planning, but increasing volumes strain radiologist capacity, leading to delays and errors.
- Deep learning (DL) shows promise for medical image diagnosis, yet its reliance on labeled data restricts clinical application.
- Self-supervised learning (SSL) offers a solution to overcome data labeling challenges in medical AI.
Purpose of the Study:
- To evaluate the performance of SSL, specifically DINOv2, across diverse medical image datasets.
- To compare SSL methods against traditional supervised learning (SL) models for medical image diagnosis.
- To introduce a novel framework leveraging DINOv2 embeddings for semantic search in medical databases, enhancing clinical workflows.
Main Methods:
- Utilized DINOv2 embeddings for semantic search in medical databases via Qdrant, enabling efficient retrieval of similar cases.
- Compared the diagnostic accuracy of SSL models (DINOv2, BYOL, SimCLR) with traditional SL models on various medical image datasets.
- Integrated DINOv2 with ViT-CX, a transformer-tailored causal explanation method, to generate clinically actionable heatmaps for model interpretability.
Main Results:
- SSL, especially DINOv2, demonstrated superior diagnostic accuracy compared to traditional SL, achieving high performance across Lung cancer, brain tumor, leukemia, and eye retina disease datasets.
- The DINOv2 framework enabled efficient semantic search, allowing clinicians to retrieve similar medical cases rapidly.
- The combination of DINOv2 and ViT-CX provided interpretable heatmaps, highlighting tumor or cellular patterns, a feature lacking in previous SSL studies.
Conclusions:
- Self-supervised learning, particularly DINOv2, effectively addresses data labeling limitations in medical imaging, offering enhanced diagnostic accuracy.
- The developed framework integrates semantic search and interpretable AI, significantly improving the efficiency and reliability of medical image analysis.
- This research paves the way for more advanced, accurate, and clinically applicable AI tools in medical diagnostics.
