A multi-modal deep learning framework for enhanced breast cancer diagnosis using mammograms and clinical data

The fatality due to breast cancer is one of the most universal causes among women worldwide. The limited interpretation is due to tissue overlapping, high breast density and delicate lesion characteristics even though mammography is the standard screening tool. To overcome this universal challenges, we propose a multi-model deep learning framework that integrates mammographic image with structured clinical data (note: synthetic clinical variables were employed based on the known distributions of the risk factors due to the lack of real paired data) to achieve the accuracy in the diagnostics. Architectural developments establish promising directions for future multi-modal tactic with reliable clinical data. The architecture utilize a hierarchical fusion approach that incorporates with cross model attention mechanism and dynamic modality weighting to efficiently integrate heterogeneous features. Moreover, a region of interest care module is integrated to highlight crucial anatomical regions in mammograms. Experimental evaluations on a benchmark dataset validate a classification accuracy of 94.6% (95% CI: 92.8-96.4%) and an AUC of 0.963 (95% CI: 0.951-0.975), expressively overtaking prevailing baseline models (p < 0.01). These results suggested the significant potential of projected architectural framework as a proof-of-concept for steadfast and easily explainable clinical decision supporting tools.