Generalization Evaluation of Machine Learning Numerical Observers for Image Quality Assessment.

Area of Science:

• Medical Imaging
• Machine Learning
• Image Quality Assessment

Background:

• Human observer (HumO) studies are the gold standard for medical image evaluation but are impractical for early development.
• Numerical observers (NOs) serve as surrogates for HumOs, with the channelized Hotelling observer (CHO) being widely used.
• Previous work established that NO development is a machine learning problem, leading to the channelized support vector machine (CSVM) observer.

Purpose of the Study:

• To develop and evaluate two new machine learning-based numerical observers (NOs) for predicting human observer performance in cardiac SPECT image analysis.
• To improve upon existing NOs by reducing model complexity and computation time while maintaining high accuracy.
• To assess the generalization performance of the proposed NOs in a practical scenario involving different image reconstruction methods.

Main Methods:

• Developed two novel regression models within a Bayesian machine-learning framework: channelized relevance vector machine (CRVM) and multi-kernel CRVM (MKCRVM).
• Evaluated the proposed CRVM and MKCRVM against the channelized Hotelling observer (CHO) and the previously developed channelized support vector machine (CSVM) observer.
• Assessed generalization performance by training NOs on one set of reconstructed images and testing on a different set obtained via a distinct reconstruction method.

Main Results:

• The proposed CRVM and MKCRVM models achieved accuracy comparable to the CSVM method.
• Both new models significantly outperformed the widely used CHO in predicting human observer performance.
• CRVM and MKCRVM demonstrated dramatically reduced model complexity and computation time compared to previous methods.

Conclusions:

• The CRVM and MKCRVM represent advanced numerical observers that offer a more efficient and accurate alternative to existing methods for medical image quality assessment.
• These models provide a practical solution for evaluating imaging devices and algorithms in early development stages.
• The findings support the application of Bayesian machine learning for developing robust and generalizable numerical observers in medical imaging.