A Knowledge-Guided Multi-Modal Neural Network for Breast Cancer Molecular Subtyping

Area of Science:

• Oncology
• Medical Imaging
• Artificial Intelligence

Background:

• Accurate HER2 subtyping is crucial for effective breast cancer targeted therapy selection.
• Current biopsy-based methods face limitations due to tumor heterogeneity and sampling bias.
• Non-invasive assessment methods are needed to overcome biopsy limitations.

Purpose of the Study:

• To develop a knowledge-guided multi-modal neural network (KMNet) for non-invasive HER2 subtyping.
• To integrate clinical data and ultrasound images for improved diagnostic accuracy.
• To provide a potential tool for clinical decision support in breast cancer treatment.

Main Methods:

• Proposed KMNet integrates clinical data and ultrasound images using a hybrid approach.
• A Graph-based Clinical Feature encoder (GCF) with Graph Convolutional Network (GCN) models clinical indicator relationships.
• A Convolutional Neural Network (CNN) and Vision Transformer (ViT)-based hybrid image encoder (CVUIF) extracts image features.
• A Reduced Dimensional Fusion (RDF) module combines multi-modal features for HER2 subtyping.

Main Results:

• KMNet demonstrated superior performance in HER2 subtyping compared to state-of-the-art multi-modal algorithms.
• Experiments were validated on private (HER2USC) and public (BCW, BCa, SIIM-ISIC) datasets.
• The model effectively integrates diverse data sources for accurate subtyping.

Conclusions:

• KMNet offers a promising non-invasive approach for HER2 subtyping in breast cancer.
• The method has the potential to enhance clinical decision-making for targeted therapies.
• This AI-driven approach addresses limitations of traditional biopsy methods.