Improving the reliability of diagnostic tests in population-based agreement studies

Area of Science:

• Biostatistics
• Medical Imaging Analysis
• Health Services Research

Background:

• Assessing diagnostic procedure reliability, such as mammography for breast cancer detection, is crucial.
• Large-scale studies with numerous raters and subjects present challenges in measuring inter-rater agreement.
• Identifying factors influencing rater discrepancies is key to enhancing diagnostic reliability.

Purpose of the Study:

• To extend existing agreement models for binary ratings to incorporate covariate information.
• To develop a flexible statistical framework for analyzing large-scale diagnostic studies.
• To enable the assessment of agreement considering subject and rater characteristics.

Main Methods:

• Utilized generalized linear mixed models with a probit link function.
• Extended a population-based agreement model to include fixed and/or random effects for covariates.
• Developed methods to assess agreement between specific subgroups of raters and subjects.

Main Results:

• The proposed model effectively incorporates covariate information into agreement assessment.
• Demonstrated the ability to compare agreement across different rater or subject subgroups.
• Simulation studies confirmed the performance of the developed models and measures.

Conclusions:

• The extended model provides a robust approach for analyzing agreement in large-scale diagnostic studies.
• Incorporating covariates enhances the understanding of factors affecting diagnostic reliability.
• The methodology is applicable to real-world scenarios, such as breast cancer screening.