Incorporating modality-specific intensity prior as text prompt for multimodal myocardial pathology segmentation.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Cardiology

Background:

• Accurate myocardial pathology segmentation from multimodal cardiac magnetic resonance imaging (CMR) is essential for myocardial infarction risk assessment and treatment planning.
• Traditional segmentation methods often rely on morphological and geometrical priors, which may limit their effectiveness.
• There is a need for advanced segmentation techniques that can leverage multimodal information more effectively.

Purpose of the Study:

• To propose a novel intensity-guided multimodal image segmentation (I-MMSeg) method for enhanced myocardial pathology segmentation.
• To leverage multimodal large language models (LLMs) for generating modality-specific intensity priors to guide the segmentation process.
• To improve the accuracy and robustness of myocardial pathology segmentation in multimodal CMR.

Main Methods:

• Developed an intensity-guided multimodal image segmentation (I-MMSeg) method.
• Utilized multimodal large language models (e.g., GPT-4o) to generate modality-specific intensity priors capturing relative intensity orders and boundary characteristics.
• Employed a CLIP-based model to encode intensity priors and incorporated them via an intensity-prior-guided cross-modal feature enhancement module and a class feature modulation module.

Main Results:

• The proposed I-MMSeg method demonstrated superior performance compared to state-of-the-art myocardial pathology segmentation approaches.
• The intensity priors generated by LLMs effectively guided the segmentation process, enhancing feature discriminability.
• Experimental results validate the efficacy of the proposed modules in improving segmentation accuracy.

Conclusions:

• The I-MMSeg method offers a significant advancement in myocardial pathology segmentation using multimodal CMR.
• The integration of LLM-generated intensity priors provides effective high-level guidance for segmentation.
• This approach holds promise for improving clinical decision-making in myocardial infarction management.