A novel graphical evaluation of agreement

Area of Science:

• Biostatistics
• Medical Imaging
• Clinical Measurement

Background:

• Bland-Altman plots and limits of agreement are standard for assessing measurement reproducibility.
• Inconsistencies arise when using limits of agreement with relative indices like the concordance correlation coefficient (CCC).
• Large data variance can lead to misleading interpretations of agreement with traditional methods.

Purpose of the Study:

• To develop a novel, CCC-based graphical guidance for evaluating reproducibility and reliability.
• To address the inconsistency between limits of agreement and scaled indices like CCC.

Main Methods:

• Utilized the concordance correlation coefficient (CCC) to construct a reference band for two measurements.
• Conducted simulation studies and applied the method to real-world data (peak expiratory flow, Radiomics).

Main Results:

• Demonstrated that limits of agreement can be inconsistent with CCC when an acceptable difference is not defined.
• The proposed CCC-based graphical evaluation method shows consistency with CCC results.
• Real data applications confirm the utility of the new graphical approach.

Conclusions:

• A novel scaled index-based guidance, consistent with CCC, is proposed for graphical evaluation of reproducibility.
• This CCC-based method offers potential advantages over limits of agreement in relevant settings.
• The new guidance enhances the graphical assessment of measurement reliability.