Prioritizing cases from a multi-institutional cohort for a dataset of pathologist annotations

Area of Science:

• Oncology
• Computational Pathology
• Artificial Intelligence

Background:

• Standardized validation datasets are crucial for comparing artificial intelligence and machine learning (AI/ML) models in cancer research.
• Stromal tumor-infiltrating lymphocytes (sTILs) are important prognostic markers in triple-negative breast cancer (TNBC).
• Developing robust AI/ML models for sTILs assessment requires high-quality, diverse validation data.

Purpose of the Study:

• To create a comprehensive validation dataset for AI/ML models assessing sTILs in TNBC.
• To implement a novel case prioritization method to ensure representation of diverse patient subgroups.
• To facilitate direct comparison of AI/ML model performance across different research groups.

Main Methods:

• Obtained whole slide images (WSIs) and clinical metadata for TNBC core biopsies from two academic medical centers.
• Selected regions of interest (ROIs) targeting diverse tissue morphologies and sTILs densities.
• Implemented a hierarchical rank-sort method for case prioritization, focusing on underrepresented clinical factors.

Main Results:

• Compiled data from 122 glass slides across 105 unique TNBC patients.
• Improved the skewness of sTILs density distribution from 0.60 to 0.46 through case prioritization.
• Increased the entropy of sTILs density bins from 1.20 to 1.24, enhancing data diversity.

Conclusions:

• The developed prioritization method effectively enhances the representation of underrepresented patient subgroups in the validation dataset.
• This approach is vital for creating a robust and equitable dataset for training and validating AI/ML models in TNBC research.
• The methodology described facilitates the creation of pivotal studies for AI/ML model development in digital pathology.