The NCI Imaging Data Commons as a platform for reproducible research in computational pathology

Area of Science:

• Computational pathology
• Machine learning
• Cancer imaging

Background:

• Reproducibility is a key challenge in computational pathology (CompPath) research.
• The NCI Imaging Data Commons (IDC) offers over 120 FAIR-compliant cancer image collections.
• IDC is designed for integration with cloud machine learning (ML) services.

Purpose of the Study:

• To evaluate the potential of the NCI Imaging Data Commons (IDC) in improving reproducibility for CompPath research.
• To explore the use of IDC datasets and cloud ML services for consistent ML model development.

Main Methods:

• Implemented two experiments using a representative ML-based method for lung tumor tissue classification.
• Trained and/or evaluated ML models on different datasets within the IDC.
• Assessed reproducibility by running experiments multiple times with identically configured cloud ML service instances.

Main Results:

• Experiments demonstrated a high degree of reproducibility across multiple runs.
• Minor variations in Area Under the Curve (AUC) values were observed, indicating practical limits to reproducibility.
• The NCI Imaging Data Commons facilitated consistent dataset access and computational environments.

Conclusions:

• The IDC significantly aids in achieving reproducibility in CompPath research.
• Reusing identical datasets from the IDC is crucial for reproducible results.
• Integration with cloud ML services enables standardized computational environments for CompPath studies.