Hybrid diagnostic framework for bone cancer detection using deep learning and radiomics analysis

Currently, bone cancer remains a big challenge in healthcare, early and accurate diagnosis is therefore key to achieving the required treatment outcomes. To this end, this research attempts to present a novel hybrid framework, i.e. TriMedNet, which works to classify bone cancer using multi-modal data sources. The diagnostic model is derived from integrating imaging data (MRI scans), unstructured clinical note texts and structured patient metrics (e.g., blood pressure, glucose levels). The three specialized branches of TriMedNet are; a Convolutional Neural Network (CNN) for image feature extraction, a Transformer based encoder Bidirectional Encoder Representations from Transformers (BERT) for text analysis, and fully connected dense layers for dealing with numerical data. The features extracted from each branch are fused and sent to the last classification layer for tumor diagnosis. The model was trained and evaluated on publicly available Roboflow dataset, with biopsy as well as blood test results. The accuracy of TriMedNet is 98.5%, precision 97.6%, recall 98.2, which effectively support the clinical decision making of bone cancer diagnosis. In research confirm that when multi-modal features are fused together, the diagnostic performance is greater than that of single modality approaches.