Perception without preconception: comparison between the human and machine learner in recognition of tissues from...

Area of Science:

• Histopathology
• Computer Vision
• Medical Education

Background:

• Deep neural networks (DNNs) excel at image classification but their performance is often benchmarked against perfect human accuracy.
• Histological image classification presents a unique challenge where human accuracy is not perfect, allowing for direct comparison with machine learning (ML) models.
• Comparing human learners and DNNs on novel datasets, like histology, can reveal insights into their respective learning and generalization capabilities.

Purpose of the Study:

• To compare the performance of two deep neural network models (VGG16 and Inception V2) against medical students in classifying histological images.
• To investigate whether ML models exhibit similar error patterns to human learners when classifying images within and outside their training scope.
• To explore the differences between machine 'training' and human 'learning' in the context of image classification.

Main Methods:

• Developed two ML models (VML and IML) using VGG16 and Inception V2 architectures with transfer learning for a 10-class histological image classifier.
• Trained ML models on 700 histological images and validated on 300 images, then tested on a separate set of 100 images.
• Administered online quizzes to 66 medical students using the same validation set and an additional set of out-of-training-scope (OTS) images, comparing their performance to the ML models.

Main Results:

• ML models achieved high accuracy (85.67% for VML, 89% for IML) on the validation set, significantly outperforming medical students (55.14% accuracy) on the initial quiz.
• ML models showed accuracy between 91-93% when tested on the same quiz questions as students, indicating superior performance within the training scope.
• Medical students outperformed ML models on OTS images, demonstrating better generalization and pattern recognition on novel data, suggesting 'training' differs from 'learning'.

Conclusions:

• Within the training scope, ML models surpass the performance of most medical students in histological image classification.
• Students achieving accuracy comparable to ML models exhibited similar error profiles, suggesting shared feature extraction mechanisms.
• Humans demonstrate superior ability to generalize and recognize patterns in novel datasets (out-of-scope images) compared to ML models, highlighting the adaptive nature of human learning.