Fusion of X-Ray Images and Clinical Data for a Multimodal Deep Learning Prediction Model of Osteoporosis: Algorithm...

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Bone Health

Background:

• Osteoporosis is a critical bone disease characterized by reduced bone mineral density, increasing fracture risk.
• Artificial intelligence (AI) offers potential for analyzing bone structure and fusing multimodal data for improved osteoporosis diagnosis.

Purpose of the Study:

• To develop and evaluate a multimodal AI model integrating chest X-rays and clinical data for opportunistic osteoporosis screening.
• To compare the performance of this multimodal model against existing methods.

Main Methods:

• Utilized multimodal data from 1780 patients, including chest X-ray images and clinical parameters.
• Developed a convolutional neural network (CNN) with transfer learning for image analysis.
• Employed a gradient-based wavelet feature extraction method with an attention mechanism for enhanced feature fusion.

Main Results:

• The multimodal AI model significantly outperformed traditional methods across key metrics: AUC (0.975 vs. 0.951), accuracy (92.36% vs. 89.32%), sensitivity (91.23% vs. 89.82%), and specificity (93.92% vs. 88.64%).
• The model demonstrated substantial improvements in all four evaluation metrics compared to using X-ray images alone.

Conclusions:

• The developed multimodal AI model shows superior performance for osteoporosis screening compared to unimodal approaches and traditional methods.
• Limitations include dataset size and retrospective design; future research should focus on larger, diverse datasets and external validation for broader clinical applicability.