Cross-modal similar clinical case retrieval using a modular model based on contrastive learning and k-nearest...

Area of Science:

• Medical Informatics
• Artificial Intelligence in Medicine
• Clinical Decision Support

Background:

• Electronic health records enable case-based clinical decision-making.
• Existing similar case retrieval methods primarily use single-modal data.
• Cross-modal clinical case retrieval research is limited.

Purpose of the Study:

• To develop a CRoss-Modal Retrieval (CRMR) model.
• To enable retrieval of similar clinical cases across different data modalities.

Main Methods:

• Utilized the Medical Information Mart for Intensive Care-Chest X-ray (MIMIC-CXR) dataset.
• Developed a modular CRMR model with feature extraction, transformation, and retrieval components.
• Employed contrastive deep learning and k-nearest neighbor search for cross-modal retrieval.

Main Results:

• Achieved average retrieval precision (AP@k) ranging from 76.3% to 77.9%.
• Demonstrated rapid retrieval times (0.013–0.016 ms).
• Successfully retrieved cases with matching radiographic manifestations.

Conclusions:

• The CRMR model shows strong cross-modal retrieval performance for clinical case analysis.
• The model has potential for scalability and handling diverse data types.
• CRMR can assist clinicians in making optimal and explainable decisions.