Uncertainty-Aware Medical Diagnostic Phrase Identification and Grounding

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Clinical Informatics

Background:

• Current medical phrase grounding relies on manual extraction, hindering efficiency and increasing clinician workload.
• Lack of model confidence estimation in existing methods limits clinical trust and usability.
• A novel task, Medical Report Grounding (MRG), is introduced to automate phrase-to-image linking.

Purpose of the Study:

• To develop an end-to-end framework for directly identifying diagnostic phrases and their corresponding grounding boxes from medical reports.
• To enhance the efficiency, reliability, and clinical trust in medical phrase grounding.

Main Methods:

• Proposed uMedGround framework utilizing a multimodal large language model with a unique embedded token ($\lt $$\mathtt {BOX}$$\gt $) for enhanced detection.
• Employed a vision encoder-decoder to process the embedded token and image for generating grounding boxes.
• Incorporated an uncertainty-aware prediction model to improve grounding prediction robustness and reliability.

Main Results:

• uMedGround demonstrated superior performance compared to state-of-the-art medical phrase grounding methods and fine-tuned large visual-language models.
• The framework achieved high effectiveness and reliability in experimental evaluations.
• The study represents the first exploration of the MRG task.

Conclusions:

• uMedGround offers a robust and reliable solution for the novel MRG task, directly linking diagnostic phrases to medical images.
• The framework's uncertainty-aware predictions enhance clinical trust and usability.
• uMedGround shows promise for applications in medical visual question answering and class-based localization, aiding clinical interpretation.