Toward accurate breast cancer classification: A review of multi-modal machine learning approaches

The innovations in classifying breast cancer into malignant and benign categories and further categorizing it into molecular subtypes have reshaped healthcare services, enabling accurate diagnosis of these complex conditions. Identification of molecular subtypes of breast cancer is one of the most important treatment challenges, as these subtypes can have an enormous effect on the prognosis and treatment approaches. Data integration from various modalities, such as transcriptomics, imaging, and genomics, has been crucial in leveraging new opportunities to increase classification accuracy and improve individualized treatment plans. These heterogeneous data sources are examined by applying deep learning algorithms, which provide further insights into the complex patterns that traditional approaches often overlook. In this paper, we explore the various modalities researchers use to investigate breast cancer and the intriguing fusion techniques employed to combine these modalities. We also review the most recent models (traditional, machine learning, and deep learning), emphasizing their improvements over traditional classification methods and the molecular subtype categorization of breast cancer. Furthermore, the emphasis of this review is to examine techniques to process the entire image of the breast tissue slide, which is challenging, particularly due to its size. We explore recent advances in multiple instance learning tasks and the use of attention-based transformers and similar architectures for annotating the WSI slides before using them for cancer classification. We additionally discuss the interpretability tools-attention maps, saliency maps and model explainability- in the context of transformers. In a nutshell, we aim to provide an in-depth look at the revolutionary capabilities of deep learning models in precision oncology and guide future research paths in this crucial field by synthesizing existing studies.