Classifying the molecular subtype of breast cancer using vision transformer and convolutional neural network features

Identification of the molecular subtypes in breast cancer allows to optimize treatment strategies, but usually requires invasive needle biopsy. Recently, non-invasive imaging has emerged as promising means to classify them. Magnetic resonance imaging is often used for this purpose because it is three-dimensional and highly informative. Instead, only a few reports have documented the use of mammograms. Given that mammography is the first choice for breast cancer screening, using it to classify molecular subtypes would allow for early intervention on a much wider scale. Here, we aimed to evaluate the effectiveness of combining global and local mammographic features by using Vision Transformer (ViT) and Convolutional Neural Network (CNN) to classify molecular subtypes in breast cancer.

The feature values for binary classification were calculated using the ViT and EfficientnetV2 feature extractors, followed by dimensional compression via principal component analysis. LightGBM was used to perform binary classification of each molecular subtype: triple-negative, HER2-enriched, luminal A, and luminal B.

The combination of ViT and CNN achieved higher accuracy than ViT or CNN alone. The sensitivity for triple-negative subtypes was very high (0.900, with F-value = 0.818); whereas F-value and sensitivity were 0.720 and 0.750 for HER2-enriched, 0.765 and 0.867 for luminal A, and 0.614 and 0.711 for luminal B subtypes, respectively.

Features obtained from mammograms by combining ViT and CNN allow the classification of molecular subtypes with high accuracy. This approach could streamline early treatment workflows and triage, especially for poor prognosis subtypes such as triple-negative breast cancer.