Evaluation of AI-Based Medical Device Concerning Localization Information Using Nonparametric Inference for the...

Area of Science:

• Medical Imaging Analysis
• Statistical Methods in Diagnostics
• Biostatistics

Background:

• Alternative free-response receiver operating characteristic (AFROC) curves are widely used for diagnostic test performance evaluation, particularly for lesion detection and localization.
• Current AFROC inference methods often depend on untestable assumptions of data independence and specific parametric models, limiting their practical applicability.
• The limitations of existing parametric approaches necessitate the development of more robust and flexible statistical methods for AFROC analysis.

Purpose of the Study:

• To propose and validate novel nonparametric inference methods for the AFROC curve.
• To develop a bootstrap-based approach for constructing confidence intervals and bands for AFROC curves and related indices.
• To demonstrate the practical utility of the proposed nonparametric methods in real-world diagnostic scenarios.

Main Methods:

• Development of nonparametric inference techniques for the AFROC curve, including derivation of asymptotic properties for the empirical AFROC curve.
• Introduction of a novel bootstrap methodology to generate confidence intervals for AFROC-related indices and confidence bands for the AFROC curve.
• Comparative analysis through simulations to assess the performance of the proposed nonparametric method against existing parametric approaches.

Main Results:

• The proposed nonparametric inference method for AFROC curves demonstrates superior performance compared to traditional parametric methods, especially when parametric assumptions are not met.
• The bootstrap method effectively constructs reliable confidence intervals and bands, enhancing the interpretability of diagnostic test performance.
• Simulations confirm the robustness and accuracy of the nonparametric approach under various conditions.

Conclusions:

• The developed nonparametric methods provide a more reliable and flexible framework for analyzing AFROC curves in diagnostic test evaluation.
• This approach overcomes the limitations of parametric assumptions, offering improved accuracy in assessing lesion detection and localization performance.
• The method is practically applicable, as shown by its use in evaluating an AI-assisted pulmonary nodule diagnosis system.