[A new method for agreement evaluation based on AC₁].

Area of Science:

• Biostatistics
• Medical Research Methodology

Background:

• Assessing inter-rater or inter-method agreement is crucial in medical studies.
• Existing agreement coefficients, such as kappa, have known limitations and paradoxes.
• There is a need for more accurate and reliable statistical methods for agreement assessment.

Purpose of the Study:

• To propose an alternative statistical method, CEA (Chance-corrected Agreement), for assessing agreement in binary data.
• To evaluate the performance of CEA by comparing it with kappa and AC1 coefficients.
• To investigate the impact of prevalence rate and chance agreement on total agreement.

Main Methods:

• Development of the CEA statistic, an adjustment of the AC1 statistic.
• Simulation studies to explore the behavior of kappa, AC1, and CEA under varying conditions.
• Real data analysis to validate the findings from simulations.

Main Results:

• The proposed CEA method demonstrates stability and reliability in assessing agreement for binary data.
• CEA effectively adjusts for chance agreement, addressing limitations of other coefficients.
• Simulations and real data analysis confirmed CEA's superior performance compared to kappa and AC1 in certain scenarios.

Conclusions:

• CEA offers a robust and reliable alternative for measuring agreement in medical research.
• The method is particularly useful for binary data where chance agreement can be a confounding factor.
• CEA enhances the accuracy and dependability of agreement assessments in clinical and research settings.