Digital pathology and multimodal learning on oncology data

Cancer presents a complex tapestry of biological, clinical, and molecular characteristics that collectively influence its diagnosis, progression, and treatment. This review article delves into the recent advancements in integrating multimodal oncology data, a pivotal aspect in the evolving field of digital pathology (DP). The complexity and heterogeneity of cancer, characterized by its multimodal nature, present significant challenges in diagnosis and treatment. Traditional methods of oncology data analysis often fall short of capturing the comprehensive landscape of cancer's multifaceted characteristics. The advent of artificial intelligence, machine learning, and deep learning has marked a new era in multimodal learning. These technologies have revolutionized how oncologists and researchers understand and approach cancer, allowing for a more nuanced and detailed analysis. In this review article, we attempt to examine and present how DP enriches its methods and analysis with other data modalities, including clinical, radiological, and molecular information. We present opportunities and challenges of multimodal learning in oncology, highlighting the synergistic potential of combining various data types for improving cancer care with a focus on DP. Continuous innovation in multimodal data integration will be instrumental in transforming cancer diagnosis, treatment planning, prognosis, and post-treatment surveillance.