Related Experiment Video
Updated: Jun 29, 2026
04:17
DNA Virus Detection System Based on RPA-CRISPR/Cas12a-SPM and Deep Learning
Published on: May 10, 2024
Intelligent Attention-Driven Deep Learning for Hip Disease Diagnosis: Fusing Multimodal Imaging and Clinical Text for
Jinming Zhang1,2, He Gong1,2, Pengling Ren1,2
1Medical Engineering & Engineering Medicine Innovation Center, Hangzhou International Innovation Institute, Beihang University, Hangzhou 311115, China.
Medicina (Kaunas, Lithuania)
|February 27, 2026
Summary
Multimodal deep learning integrating clinical data, X-rays, and CT scans significantly enhances the early detection and classification of hip joint disorders like osteoarthritis and osteonecrosis of the femoral head (ONFH). This approach offers improved diagnostic accuracy.
Area of Science:
- Orthopedic imaging analysis
- Artificial intelligence in medicine
- Deep learning for medical diagnosis
Background:
- Hip joint disorders present overlapping radiological features, challenging single-modality imaging and early diagnosis.
- Isolated imaging or clinical data may not fully capture disease-specific pathological characteristics.
Purpose of the Study:
- To develop and evaluate a multimodal deep learning framework for improved diagnosis of hip joint disorders.
- To assess the diagnostic performance of integrating radiographs, CT volumes, and clinical texts.
Main Methods:
- A retrospective study of 605 hip joints (normal, osteoarthritis, osteonecrosis of the femoral head (ONFH), femoroacetabular impingement (FAI)) from Center A, with external validation on 24 hips from Center B.
- Development of a deep learning framework using ResNet50, 3D-ResNet50, and BERT for feature extraction.
- Attention-based fusion of multimodal features for four-class classification.
Main Results:
- The combined Clinical+X-ray+CT model achieved an AUC of 0.949, outperforming single-modality models.
- Consistent improvements in accuracy, sensitivity, and specificity were observed.
- Grad-CAM visualizations confirmed the model's focus on clinically relevant anatomical regions.
Conclusions:
- Attention-based multimodal feature fusion significantly enhances diagnostic performance for hip joint diseases.
- The developed framework is interpretable and clinically applicable for early detection and precise classification in orthopedic imaging.
