Related Experiment Video
Updated: Jun 16, 2025
10:06
High-resolution Functional Magnetic Resonance Imaging Methods for Human Midbrain
Published on: May 10, 2012
12.9K
Task-based assessment for neural networks: evaluating undersampled MRI reconstructions based on human observer signal
Joshua D Herman1, Rachel E Roca1, Alexandra G O'Neill1
1Manhattan College, Department of Mathematics, The Bronx, New York, United States.
Journal of Medical Imaging (Bellingham, Wash.)
|August 15, 2024
Summary
Conventional metrics overestimated undersampling rates in neural network-reconstructed MRI. Human observers preferred lower acceleration (2×) than metrics (3×) for detecting subtle signals, highlighting the need for task-based image quality assessment.
Area of Science:
- Medical Imaging
- Artificial Intelligence
- Image Reconstruction
Background:
- Neural networks are increasingly used for reconstructing undersampled magnetic resonance imaging (MRI).
- Assessing image quality in these reconstructions is challenging due to complex artifacts.
- Task-based approaches are needed to evaluate image quality relevant to specific diagnostic tasks.
Purpose of the Study:
- To compare conventional global quantitative metrics with human observer performance for evaluating image quality in neural network-reconstructed undersampled MRI.
- To determine the optimal undersampling acceleration (2×, 3×, 4×, 5×) based on both conventional metrics and human detection task performance.
Main Methods:
- Reconstruction of undersampled MRI using a U-Net with 1D undersampling rates (2× to 5×).
- Evaluation using conventional metrics: normalized root mean squared error (NRMSE) and structural similarity (SSIM).
- Comparison with a task-based assessment using a two-alternative forced choice (2-AFC) observer study for detecting a subtle signal.
Main Results:
- Human observers in the 2-AFC study selected 2× undersampling as optimal.
- Conventional metrics (SSIM and NRMSE) favored 3× undersampling.
- This discrepancy was observed for both SSIM and MSE loss functions across different training set sizes.
Conclusions:
- Conventional metrics like SSIM and NRMSE overestimate achievable undersampling rates when evaluated against human performance in subtle signal detection tasks.
- Task-based image quality assessment is crucial for accurately determining optimal undersampling in neural network-based MRI reconstruction.
- A steep decline in image quality occurs between 2× and 3× undersampling, which is better identified by human observers than global metrics.
