How accurately can quantitative imaging methods be ranked without ground truth: An upper bound on no-gold-standard...

Area of Science:

• Medical Imaging
• Statistical Evaluation
• Quantitative Imaging

Background:

• Objective evaluation of quantitative imaging (QI) methods is crucial but often limited by the absence of gold standards.
• No-gold-standard evaluation (NGSE) techniques have emerged to rank QI methods without ground truth data.
• The accuracy achievable in ranking QI methods using NGSE remains an open question.

Approach:

• A Cramer-Rao bound (CRB)-based framework is proposed to establish an upper bound on the accuracy of ranking QI methods without ground truth.
• This framework is applied to guide the utilization of the regression-without-truth (RWT) NGSE technique.
• The framework's utility is demonstrated by quantifying the performance of the RWT technique across varying patient cohort sizes.

Key Points:

• The proposed CRB-based framework provides a theoretical upper limit for ranking QI methods when ground truth is unavailable.
• The framework effectively quantifies the performance of NGSE techniques, such as RWT.
• Performance quantification is shown to be dependent on the number of patients included in the evaluation.

Conclusions:

• The CRB-based framework offers a method to understand the inherent limitations in ranking QI methods without gold standards.
• This approach aids in optimizing the application of NGSE techniques by providing performance benchmarks.
• The findings encourage further exploration of this framework for diverse applications in medical imaging evaluation.