An Adaptive Fusion Network for Breast Tumor Grading Based on Graph Structure Learning

Breast cancer is a serious threat to women's life and health. Early grading of breast tumor malignancy is the key for improving patient survival rate. In clinical diagnosis, physicians often utilize multi-modal data information to accurately grade. As a new multi-modal data processing method, graph convolutional network is expected to improve the grading accuracy of breast tumors. To solve the problem that a single graph cannot effectively utilize multi-modal information, this paper proposes a multi-graph adaptive fusion network to perform accurate breast tumor grading by using multi-modal data. Firstly, a multi-modal information extraction network is designed to extract specific modal information and graph structure from multi-modal data, which is composed of graph structure learning layer and graph auto-encoder. Secondly, by calculating the attention scores of different modalities as weight coefficients, the modal aggregation features, and modal aggregation graphs are obtained by fusing in a weighted manner. Finally, a neighborhood adaptive aggregation module is designed to gather multi graphs and optimize the conventional graph convolution aggregation mode by calculating the neighborhood aggregation coefficient. Experimental results on both public and private breast tumor datasets show that the graph learning-based diagnosis model can effectively process multi-modal medical data and improve the accuracy of multi grading for breast tumors.