Critical issues concerning the assessment sets in agreement assessment

Area of Science:

• Statistics
• Biostatistics
• Medical Statistics

Background:

• The assessment set is crucial for agreement assessment, building on Bland and Altman's work.
• Understanding the assessment set is key to accurate statistical analysis and method comparison.

Purpose of the Study:

• To explore critical issues and provide guidance on selecting and constructing assessment sets.
• To differentiate between point estimation and hypothesis testing approaches for assessment sets.
• To propose methods for constructing assessment sets when evaluating multiple methods.

Main Methods:

• Comparative analysis of point estimation and hypothesis testing in assessment set selection.
• Exploration of minimum volume principles for assessment sets, analogous to confidence intervals.
• Development and illustration of two novel methods for constructing assessment sets for multiple method assessments.

Main Results:

• Fundamental differences between point estimation and hypothesis testing approaches for assessment sets are elucidated.
• The preference for minimum volume assessment sets is established, drawing parallels with confidence interval comparisons.
• Two new methods for constructing assessment sets for multiple method agreement assessment are proposed and validated.

Conclusions:

• Proper selection and construction of assessment sets are vital for robust agreement assessment.
• Minimum volume assessment sets offer advantages in statistical power and precision.
• The proposed methods provide practical tools for researchers assessing multiple methods.