FKG-MM: A multi-modal fuzzy knowledge graph with data integration in healthcare

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare

Background:

• Accurate clinical diagnosis relies on multi-modal data (e.g., clinical records, images).
• Integrating diverse medical data sources and modalities presents significant challenges for current AI systems.
• Machine learning models struggle with the uncertainty and heterogeneity inherent in medical data.

Purpose of the Study:

• To propose a novel multi-modal fuzzy knowledge graph framework (FKG-MM) for integrating diverse medical data.
• To enhance computational performance and diagnostic accuracy by fusing multi-modal medical information.
• To address the limitations of existing methods in representing and computing with uncertain and diverse medical data.

Main Methods:

• Developed a multi-modal fuzzy knowledge graph framework (FKG-MM).
• Integrated multi-modal medical data from various sources and modalities.
• Utilized a fuzzy knowledge graph model for effective data representation and computation.
• Employed feature selection methods to combine image and tabular medical data.

Main Results:

• The FKG-MM framework demonstrated enhanced computational performance compared to unimodal data processing.
• Feature selection combining image and tabular data yielded the highest reliability in multi-modal diabetic retinopathy diagnosis.
• FKG-MM achieved a 12-14% accuracy increase by integrating image and tabular data versus tabular data alone.

Conclusions:

• The FKG-MM framework effectively integrates multi-modal medical data, improving computational performance.
• Combining image and tabular data through feature selection significantly enhances diagnostic reliability.
• FKG-MM offers a promising approach for advancing AI-driven multi-modal medical diagnosis.