Performance evaluation of deep learning models for image analysis: Considerations for visual assessment and...

Area of Science:

• Veterinary pathology
• Computational pathology
• Medical imaging analysis

Background:

• Deep learning-based automated image analysis (DL-AIA) is increasingly used in pathology for tasks like feature quantification.
• DL-AIA applications are expanding from research to diagnostic and toxicologic pathology.
• Ensuring DL-AIA safety and reliability necessitates robust generalization performance assessment.

Purpose of the Study:

• To review current performance assessment practices for DL-AIA in veterinary pathology.
• To compare visual and statistical performance assessment methods.
• To discuss considerations for rigorous statistical performance evaluation.

Main Methods:

• Identified two main approaches: visual assessment and statistical assessment.
• Statistical assessment involves a hold-out test set with ground truth labels.
• Reviewed practices including metric selection, dataset composition, ground truth quality, and resampling.

Main Results:

• Visual assessment involves subjective evaluation of predictions.
• Statistical assessment provides objective, quantitative performance metrics.
• Both methods have complementary strengths and weaknesses.

Conclusions:

• A combined approach of visual and statistical evaluation provides the most comprehensive insight into DL-AIA model performance.
• This integrated approach aids in identifying model errors and improving reliability.
• Rigorous assessment is critical for the safe and effective implementation of DL-AIA in pathology.