Related Experiment Video
Updated: Apr 28, 2026
Computer-assisted Large-scale Visualization and Quantification of Pancreatic Islet Mass, Size Distribution and Architecture
Published on: March 4, 2011
Virtual multiplex staining of the pancreatic islets across type 1 diabetes progression using a Schrödinger bridge
Yu Shen1,2, Won June Cho1,2, Saurabh Joshi1,2
1Department of Chemical & Biomolecular Engineering, Johns Hopkins University, Baltimore, MD.
Insights
Schrödinger-bridge for Multiplex ImmunoLabel Estimation (SMILE) uses diffusion models to accurately convert H&E stains to multiplex IHC images. This AI approach enhances proteomic inference from archival tissue, outperforming GANs in accuracy and stability.
Area of Science:
- Digital pathology
- Artificial intelligence in medicine
- Computational biology
Background:
- Hematoxylin and eosin (H&E) staining reveals tissue morphology but lacks molecular information.
- Immunohistochemistry (IHC) provides molecular details but is complex, costly, and time-consuming, especially for multiplex IHC (mIHC).
- Generative adversarial networks (GANs) have been explored for H&E to IHC stain conversion but suffer from instability and accuracy issues.
Purpose of the Study:
- To introduce and validate a novel diffusion model-based approach, SMILE (Schrödinger-bridge for Multiplex ImmunoLabel Estimation), for accurate H&E to mIHC stain conversion.
- To demonstrate SMILE's superiority over GANs in preserving structural integrity and molecular information during image translation.
- To establish a scalable pipeline for high-throughput proteomic inference from archival H&E slides.
Main Methods:
- Development of SMILE, a Schrödinger-bridge diffusion model, for direct mapping from H&E to mIHC images, bypassing intermediate Gaussian noise.
- Generation of a large, high-fidelity dataset of H&E-mIHC image pairs from pancreatic organ donors with diverse characteristics (diabetes status, anatomical location, age, sex).
- Comprehensive evaluation of SMILE against GANs using quantitative metrics (texture, distribution, antibody-specific) and blinded pathologist reviews.
Main Results:
- SMILE demonstrated superior performance compared to GANs in H&E to mIHC stain conversion, validated by quantitative metrics and pathologist assessments.
- The model successfully generated accurate mIHC images from external H&E data and enabled whole slide image conversion.
- SMILE accurately reconstructed 3D pancreatic islet maps across different diabetes statuses and showed efficacy in diverse applications like breast cancer (HER2, Ki67) stain conversion.
Conclusions:
- SMILE offers a robust and accurate method for inferring multiplex IHC data from H&E stained slides using advanced diffusion models.
- This AI-driven stain conversion technique significantly enhances the utility of archival tissue for proteomic analysis and digital pathology.
- The SMILE framework has transformative potential for accelerating research in pancreatic diseases and improving diagnostic capabilities.
Abstract:
Classical hematoxylin and eosin (H&E) staining enables review of tissue morphology but lacks information regarding the molecular state of cells. Immunohistochemical (IHC) techniques label specific proteins in tissue, allowing differentiation of relevant structures that may go undetectable in H&E. However, the IHC process is complex, expensive, and time-consuming, especially for multiplex IHC (mIHC) limiting its use in large cohorts. Stain conversion of H&E to IHC using generative artificial intelligence models such as generative adversarial networks (GANs) represent one solution to this problem. However, GANs are unstable during out of distribution sampling and are prone to hallucinations or mode collapse, limiting their accuracy in challenging image conversion tasks. To address this, the field has recently turned to diffusion models. Here, we introduce Schrödinger-bridge for Multiplex ImmunoLabel Estimation (SMILE). Unlike conventional diffusion models that map from source to target through an intermediate Gaussian noise, Schrödinger-bridge diffusion models skip this step and have been shown to better preserve structures during image translation. To test the performance of SMILE, we generated a large cohort of high-fidelity H&E-mIHC image pairs from pancreatic organ donors, targeting insulin, glucagon, and CD3. Our dataset well-sampled across type-1 diabetes status, pancreas anatomical location, age, and sex. Using this cohort, we demonstrate the superiority of SMILE compared to GANs via a comprehensive evaluation framework incorporating texture, distribution, and antibody-specific metrics, as well as blinded pathologist reviews. We further confirmed the ability of SMILE to generate accurate mIHC images from H&Es generated at an external site, to perform whole slide image conversion, and to generate realistic three-dimensional maps of the pancreatic islets in non-diabetic, auto-antibody positive, and type-1 diabetic donor tissue. Finally, we performed stain conversion of paired H&E to HER2 and Ki67 images in breast cancer, confirming the superiority of SMILE in diverse stain conversion applications. Collectively, this framework provides a scalable pipeline for high-throughput proteomic inference from archival H&Es, providing transformative potential for pancreatic research and digital pathology.
