A Framework for Assessing Broad Sense Agreement Between Ordinal and Continuous Measurements

Area of Science:

• Statistics
• Psychometrics
• Biostatistics

Background:

• Traditional agreement studies focus on reproducibility, limiting assessments to measurements on the same continuous scale.
• Existing methods cannot evaluate the interpretation of continuous measurements within an ordinal framework.

Purpose of the Study:

• To introduce and define a new statistical concept,
• broad sense agreement,
• to extend classical agreement measures.
• To develop a natural measure for broad sense agreement and associated nonparametric estimation and inference procedures.
• To adapt the broad sense agreement framework for longitudinal data analysis.

Main Methods:

• Development of a novel measure for broad sense agreement.
• Formulation of nonparametric estimation and inference procedures with theoretical justifications.
• Extension of the methodology to accommodate longitudinal data with multiple time points.

Main Results:

• The proposed broad sense agreement measure effectively evaluates the interpretation of continuous measurements in an ordinal scale.
• Nonparametric methods provide reliable estimation and inference for the new agreement measure.
• Simulation studies confirm the method's good performance, particularly with small sample sizes.

Conclusions:

• Broad sense agreement offers a valuable extension to traditional agreement studies, enabling richer interpretation of measurement data.
• The developed methods are applicable to various fields, including mental health research, especially in longitudinal settings.
• The proposed approach enhances the assessment of agreement beyond simple reproducibility.