Assessing agreement with relative area under the coverage probability curve.

Area of Science:

• Statistics
• Biostatistics
• Measurement Science

Background:

• Traditional methods for assessing measurement agreement often rely on a single pre-specified difference.
• The coverage probability approach is a preferred index for evaluating measurement agreement in core laboratory settings.
• Existing methods may not adequately address quality control for multiple differences or summarize agreement up to a maximum difference.

Purpose of the Study:

• To propose a novel method for assessing measurement agreement using coverage probability curves.
• To introduce a new summary index, the relative area under the coverage probability curve, for overall agreement assessment.
• To enable comparison of agreement across different intra-method or inter-method/lab/observer scenarios.

Main Methods:

• Development of the coverage probability curve to visualize measurement error across a spectrum of differences.
• Proposal of the relative area under the coverage probability curve as a summary statistic for overall agreement.
• Utilizing simulation studies and a real-world blood pressure example for methodological illustration.

Main Results:

• The coverage probability curve offers a comprehensive overview of measurement error at various disagreement levels.
• The relative area under the coverage probability curve serves as a robust index for summarizing overall measurement agreement.
• The proposed method facilitates effective comparison of agreement metrics between different measurement systems or observers.

Conclusions:

• The coverage probability curve provides a more complete assessment of measurement agreement than traditional single-difference approaches.
• The relative area under the coverage probability curve is a valuable tool for quantifying and comparing overall measurement agreement.
• This methodology enhances quality control and comparative analysis in laboratory settings and beyond.