Virtual multiplex immunofluorescence identifies lymphocyte subsets predictive of response to neoadjuvant therapy

Area of Science:

• Computational pathology
• Artificial intelligence in oncology
• Biomarker discovery

Background:

• Hematoxylin and eosin (H&E) staining is a standard pathology technique but lacks cellular specificity.
• Multiplex immunofluorescence (mIF) provides spatial immune insights but is costly and complex for clinical use.
• Novel methods are needed to extract immune information from routine H&E histology.

Purpose of the Study:

• To develop and validate a deep learning tool for translating H&E images into synthetic mIF images.
• To preserve immune cell information predictive of treatment response in breast cancer.
• To create a cost-effective and scalable alternative to mIF.

Main Methods:

• A novel pipeline, mSIGHT, was developed, integrating a registration network to align H&E and mIF images.
• The pipeline outperformed standard Pix2Pix and CycleGAN models in image translation.
• Generated synthetic mIF images were evaluated using pixel-level and biological metrics, including cell density and proximity.

Main Results:

• mSIGHT successfully generated synthetic mIF images that preserved immune cell distributions and proximity metrics.
• In a cohort of 218 breast cancer cases, predicted CD8+ T cell density was significantly associated with complete response to neoadjuvant chemotherapy.
• This association was independent of established prognostic factors.

Conclusions:

• The mSIGHT pipeline offers a scalable and affordable method to generate virtual mIF images from H&E slides.
• Interpretable immune biomarkers can be derived, aiding in the personalization of neoadjuvant therapy.
• mSIGHT has the potential to enhance treatment selection and monitoring in breast cancer.