Selective multimodal deep learning for reliable breast cancer subtype classification from histopathology and genomic...

Accurate classification of breast cancer subtypes is critical for personalized treatment planning and prognostic assessment. While histopathology and RNA-seq data each provide complementary insights into tumor biology, their integration remains challenging due to heterogeneity in data modalities. This study proposes a robust multimodal framework that combines histology-based features extracted using the CTransPath vision transformer with transcriptomic profiles to improve breast cancer subtype classification and interpretability. We developed an end-to-end pipeline incorporating three learning paradigms: RNA-Only, whole-slide image (WSI)-Only, and multimodal fusion. Histological features were extracted using a pretrained CTransPath encoder at 20× magnification (0.5 µm pixel^-1) with non-overlapping 256 × 256 pixel patches, while RNA-seq features were standardized and projected into a shared latent space. We evaluated multiple fusion techniques including gated attention, cross-attention, and concatenation. To enhance transparency, attention rollout was applied to visualize feature importance across WSIs. An uncertainty-aware smart routing mechanism was introduced to selectively use RNA-Only predictions for high-confidence cases or defer to multimodal inference for ambiguous samples, optimizing computational efficiency. Evaluated on the TCGA-BRCA dataset comprising 924 patients with PAM50 molecular subtypes, our routing-based multimodal model achieved 94.93% accuracy, outperforming both unimodal baselines and fixed fusion strategies. The routing mechanism required multimodal inference for only 38.2% of samples, achieving a 3.12× computational speedup while maintaining superior accuracy. Confusion matrices revealed significant improvements in distinguishing closely related subtypes such as Luminal A and Luminal B. Attention rollout provided interpretable heatmaps localizing discriminative histological regions associated with each subtype, aligning with established pathological criteria. This study demonstrates that integrating CTransPath-derived histology features with transcriptomic profiles through confidence-aware routing offers a practical, explainable, and computationally efficient approach for breast cancer subtype classification suitable for clinical decision support systems.