MCFL: Multimodal Collaborative Fusion Learning for Hashimoto's Thyroiditis Recognition.

Area of Science:

• Medical Imaging
• Biochemistry
• Artificial Intelligence in Medicine

Background:

• Hashimoto's thyroiditis (HT) diagnosis relies on ultrasound imaging and biochemical tests, but neither is sufficient alone.
• Existing multimodal models struggle with feature dimensional imbalance between image and text data.
• Accurate HT diagnosis requires integrating diverse data modalities effectively.

Purpose of the Study:

• To propose a novel Multimodal Collaborative Fusion Learning (MCFL) approach for enhanced HT diagnosis.
• To improve the significance and specificity of biochemical indicators using ultrasound image data.
• To overcome limitations of existing multimodal models in handling feature dimensional imbalance.

Main Methods:

• Developed a novel INNet to convert ultrasound image characteristics into Local prominent inflammatory (Lpi) and Global diffuse lesion (Gdl) indicators.
• Employed a decision tree-based optimization strategy to recalibrate biochemical indicators using Lpi and Gdl.
• Utilized a deep Q-learning framework with experience replay and ϵ-greedy strategy for diagnostic guidance and model robustness.

Main Results:

• MCFL effectively enhances and recalibrates biochemical indicators with ultrasound image data.
• The approach unifies image and textual data into a single representation space.
• Achieved state-of-the-art performance on a multimodal dataset from multiple medical centers.

Conclusions:

• MCFL offers a significant advancement in multimodal learning for HT diagnosis.
• The proposed method sets a new benchmark for accuracy and robustness in diagnosing Hashimoto's thyroiditis.
• Integrating imaging and biochemical data through collaborative fusion learning is crucial for complex disease diagnosis.