Web engine for tumor pathology image retrievals on massive scales.

Area of Science:

• Digital pathology
• Computational biology
• Histopathology image analysis

Background:

• Hematoxylin and Eosin (H&E) staining is a cornerstone of clinical histopathology.
• Current whole-slide image retrieval systems lack efficiency and versatility for clinical use.

Purpose of the Study:

• To develop and evaluate the H&E Retrieval Engine (HERE) for efficient analysis of patient cases based on H&E image similarity.
• To integrate spatial transcriptomics data with H&E images for enhanced molecular pathway identification.

Main Methods:

• Developed the HERE system utilizing artificial intelligence encoding and hierarchical skip indexing for rapid searching of large whole-slide image datasets.
• Input H&E image regions to search a 21.2 terabyte database via a 12.1-gigabyte memory index.
• Paired spatial transcriptomics with H&E images to correlate molecular data with histological features.

Main Results:

• HERE demonstrated superior performance compared to existing tools in a blinded pathologist scoring study.
• The system efficiently retrieves similar H&E image regions from extensive histopathology cohorts.
• HERE successfully linked gene transcriptomics input to image features and identified associated molecular pathways.

Conclusions:

• The H&E Retrieval Engine (HERE) provides an efficient and versatile solution for H&E image retrieval in clinical practice.
• HERE facilitates the discovery of molecular pathways underlying tumor histologies by integrating imaging and transcriptomics data.
• This tool has the potential to advance digital pathology and precision medicine by enabling sophisticated image-based case analysis.