A unified medical image segmentation evaluation method combining multi-metrics and confidence

The advancement of evaluation methodologies for medical image segmentation models has not kept pace with the development of the models themselves. Current technologies encounter challenges related to complexity and uncertainty, which hinder their efficacy in guiding clinical practice. This study aims to provide a comprehensive framework for quantifying the overall capabilities of medical image segmentation models. We propose a unified approach that facilitates the simultaneous assessment of prediction accuracy and reliability. Based on monotonic rank agreement, we integrate various accuracy metrics alongside confidence levels derived from multi-organ segmentation results to compute a final comprehensive score. The obtained scores can be used for an intuitive comparison of models: models with higher scores mean both high accuracy and high reliability, making them more clinically applicable. We conducted extensive experiments on six different medical image segmentation models and compared them from four perspectives: accuracy, reliability estimation, usable region estimation, and our proposed assessment pipeline. Experimental results verify that our method delivers more interpretable quantitative metrics to assess the practical comprehensive performance of segmentation models than the single-metric methods. The code has been released on GitHub ( https://github.com/SCUT-ML-GUO/MOMAI ).