Comparing Human-Level and Machine Learning Model Performance in White Blood Cell Morphology Assessment

Area of Science:

• Hematology
• Medical Diagnostics
• Artificial Intelligence

Background:

• Growing interest in machine learning (ML) for hematology labs, especially blood cell morphology.
• Human-level performance is a key benchmark for ML in diagnostics.
• Assessing interobserver variability and human performance is crucial for validating ML tools.

Purpose of the Study:

• To evaluate interobserver variability in blood cell morphology assessment.
• To establish human-level performance benchmarks for ML.
• To compare ML model performance against human experts in identifying white blood cells.

Main Methods:

• 1000 white blood cell images were independently labeled by 10 experts.
• Interobserver variability assessed using Fleiss' kappa.
• ML model trained and tested on the same dataset; explainability metrics generated.

Main Results:

• Substantial agreement among human observers (Fleiss' kappa = 0.608).
• Human accuracy: 95% (Sensitivity 72%, Specificity 97%).
• ML model achieved comparable accuracy (95%, Sensitivity 71%, Specificity 97%) and differentiated cell components.

Conclusions:

• Human assessment of WBC morphology has inherent subjectivity, despite substantial agreement.
• ML models demonstrate comparable performance to human experts in single cell identification.
• Further research needed to align ML evaluation with clinical morphology workflows.