Using HistoQC to predict disagreement on human epidermal growth factor receptor 2 (HER2) assessment

The human epidermal growth factor receptor 2 (HER2) gene is a significant prognostic and predictive factor for breast cancer therapy response. HER2 assessment is critical for targeted therapy eligibility, but interobserver reproducibility is a well-known issue in HER2 evaluation.

The goal of our study is to create a machine learning (ML) system able to detect whole slide images (WSIs) that might cause discrepancies among observers.

We collected 132 pathology slides with double-blind HER2 evaluation and defined the agreement between observers as a binary classification: 0 for disagreement and 1 for agreement. We utilized HistoQC software to analyze and characterize the pathology slides based on a series of quality-related features. HistoQC-derived quality metrics were used to train a machine learning model (XGBoost) to predict interobserver disagreement. The dataset was randomly split into training and testing at proportions of 60%/40%, respectively.

Our model demonstrated a mean AUC of 0.86 (standard deviation, SD = 0.09) across five cross-validation runs on the training set, highlighting its predictive reliability. The AUC on the testing set was 0.81 (confidence interval, CI = [0.82-0.94]), emphasizing the model's precision in predicting whether an unseen WSI would lead to discordance.

Our study presents a machine learning model built to identify potential diagnostic disagreements in HER2 pathology evaluations. The results demonstrate a correlation between the quality of pathology slides and diagnostic outcomes. Upon proper validation, our tool could be integrated among the existing quality assurance systems used in anatomic pathology departments to improve HER2 diagnostic process.