How accurately can quantitative imaging methods be ranked without ground truth: An upper bound on no-gold-standard...

Area of Science:

• Medical Imaging
• Statistical Evaluation
• Biomedical Data Analysis

Background:

• Objective evaluation of quantitative imaging (QI) methods is crucial but often limited by the absence of gold standards.
• No-gold-standard evaluation (NGSE) techniques have emerged to rank QI methods without ground truth.
• The precision of QI method ranking using NGSE techniques requires further quantification.

Purpose of the Study:

• To develop a framework for quantifying the upper bound on the accuracy of ranking QI methods without ground truth.
• To assess the performance of NGSE techniques, specifically regression-without-truth (RWT), using this framework.
• To guide the application and interpretation of NGSE techniques in QI method evaluation.

Main Methods:

• Proposed a Cramér-Rao bound (CRB)-based framework to establish an upper limit for ranking accuracy.
• Applied the CRB framework to evaluate the regression-without-truth (RWT) NGSE technique.
• Analyzed the framework's utility across varying numbers of patient datasets.

Main Results:

• The proposed CRB-based framework effectively quantifies the upper bound for ranking QI methods without ground truth.
• Demonstrated the framework's utility in assessing the performance of the RWT technique.
• Showcased how the framework's insights vary with different patient cohort sizes.

Conclusions:

• The CRB-based framework provides a valuable tool for understanding the performance limits of NGSE techniques.
• This framework can guide the application of NGSE methods, enhancing the reliability of QI method evaluation.
• Further research into broader applications of this upper bound quantification is warranted.