Foundation Models and Multimodal Learning for Medical Data with Applications in Oncology

The rapid expansion of medical data, including radiological imaging, digital pathology, molecular profiling, electronic health records, and longitudinal clinical narratives, has created new opportunities for AI in healthcare, while also exposing the limitations of traditional unimodal modeling approaches. Multimodal learning addresses these challenges by integrating heterogeneous data sources, and recent advances in foundation models, large-scale pretrained models designed for broad task transfer, have further accelerated this shift. Together, these developments have given rise to multimodal foundation models that aim to learn generalizable, aligned representations across diverse clinical modalities. In this review, we provide an overview of multimodal foundation models in medical AI, with a particular focus on oncology. We first outline key medical data modalities and the machine learning objectives that motivate multimodal integration, including predictive, representational, and alignment-based goals. We then examine core architectural design patterns and training paradigms underlying multimodal foundation models, such as modality-specific encoders, fusion strategies, attention mechanisms, contrastive pretraining, and scalable self-supervised learning. Building on this foundation, we survey representative multimodal models across radiology, pathology, clinical text, and molecular domains. We further review applications of multimodal foundation models across the cancer care continuum, including early detection, diagnosis and subtyping, prognosis, therapy response prediction, and longitudinal survival modeling. Finally, we discuss critical challenges that limit clinical translation, including cross-institutional heterogeneity, temporal misalignment across modalities, weak supervision, interpretability, regulatory constraints, and workflow integration, and outline future directions for advancing safe and effective deployment in real-world oncology practice.