Related Experiment Video
Updated: Aug 14, 2026
04:48
Application of Deep Learning-Based Medical Image Segmentation via Orbital Computed Tomography
Published on: November 30, 2022
Quantifying User Satisfaction: Weighted Metric Approach for Evaluating Deep Learning-Based Thigh MRI Segmentations
Falko Ensle1, Ilker Özgür Koska2,3, Nina Derron4
1Diagnostic and Interventional Radiology, University Hospital of Zurich, 8091 Zurich, Switzerland.
Diagnostics (Basel, Switzerland)
|August 13, 2026
Summary
A new deep learning model accurately predicts radiologist satisfaction with MRI segmentations. This approach offers a practical alternative to manual validation, aiding the clinical adoption of automated segmentation tools.
Area of Science:
- Medical Imaging
- Artificial Intelligence
- Biomedical Engineering
Background:
- Deep learning (DL) models are increasingly used for MRI segmentation.
- Objective prediction of user satisfaction with DL segmentations is crucial for clinical adoption.
- Current validation methods for DL segmentation can be time-consuming.
Purpose of the Study:
- To develop a clinically useful benchmark signature for DL-based MRI segmentations.
- To objectively predict user satisfaction using a weighted combination of performance metrics.
- To evaluate a hybrid DL model for predicting radiologist satisfaction scores.
Main Methods:
- Analysis of MRI data from 68 patients undergoing a randomized clinical trial.
- Segmentation of thigh muscles using fat fraction maps from axial Dixon MRI.
- Qualitative scoring of DL segmentations by radiologists using a 5-point Likert scale.
- Development of a hybrid DL model (MobileNetV2 and DINOv2) to predict Likert scores.
- Correlation analysis between quantitative metrics (Dice, Hausdorff, Jaccard) and Likert scores.
Main Results:
- The DINOv2 model demonstrated superior performance in predicting Likert scores (lower MAE and RMSE).
- A weighted combination of quantitative metrics accurately predicted high user satisfaction (Likert=5) with 84.1% accuracy.
- Individual metrics showed moderate correlation with user satisfaction, with Dice and Jaccard indices for extensor intramuscular fat being most correlated.
- Near-perfect interreader agreement (κ = 0.82) was observed for Likert scores.
Conclusions:
- The developed DL-based model accurately predicts radiologist-assigned satisfaction scores for MRI segmentations.
- A weighted combination of quantitative metrics effectively predicts user satisfaction, serving as a practical alternative to manual validation.
- These findings facilitate the clinical adoption of automated segmentation tools by providing objective performance benchmarks.