Multimodal Osteoporosis Assessment using X-ray Imaging and Clinical Data with a Hybrid Deep Learning Model

Osteoporosis is a systemic skeletal disorder characterised by low bone mass. The classification typically includes three stages: normal, osteopenia, and osteoporosis. Early detection is essential for effective treatment and reducing fracture risk. This detection is possible through X-ray imaging and clinical assessments, such as measuring T-score and SoS, etc. This study aims to enhance the accuracy of osteoporosis assessment by combining both imaging and clinical data using advanced AI techniques.

A new multimodal approach is proposed for osteoporosis assessment based on imaging and clinical data. The dataset is collected from Begum Shafia Nazir Husain Trust Hospital in Chennai, as well as from various rural clinic camps organized across different regions. For X-ray image classification, a new DL model, the Dynamic Topological Convolutional Transformer (DTCT), is proposed. For clinical data processing, a hybrid model combining Self-Organising Maps (SOM) with Graph Neural Networks (GNN) is applied. The approach is evaluated on a dataset collected from the Begum Shafia Nazir Husain Trust Hospital.

The DTCT-based classification achieves the highest accuracy of 96% for multi-class classification. The SOM-combined GNN model achieves a maximum accuracy of 95% in classification. It achieves a robust framework for integrating different data modalities in medical diagnostics.

The proposed method provides high classification accuracy, improved interpretability, robust performance against noise and variations, and a scalable architecture for efficient adaptation to diverse datasets.

The proposed approach combines imaging and clinical data for an accurate osteoporosis assessment with superior performance over existing techniques.