Evaluating patient-reported outcome measurement comparability between paper and alternate versions, using the lung...

Area of Science:

• Health Services Research
• Biostatistics
• Psychometrics

Background:

• Patient-reported outcome (PRO) instruments are crucial for clinical research.
• Ensuring measurement comparability across different administration modes (e.g., paper, web, IVR, interview) is essential for data integrity.
• Crossover study designs offer a robust framework for comparing these administration types.

Purpose of the Study:

• To present a recommended four-step guide for assessing measurement comparability of PRO instruments.
• To demonstrate the application of this guide using a PRO instrument administered via multiple modes.
• To provide a framework for deciding whether to combine data from different administration versions.

Main Methods:

• A randomized crossover design was employed to administer a PRO instrument via paper, web, interactive voice response (IVR), and interview.
• Data collected included item-level responses, mean scale scores, usability, and administration preferences.
• Statistical analyses involved item-level agreement (kappa statistics) and score comparability (intraclass correlation coefficients).

Main Results:

• Item-level agreement between paper and alternate versions showed good to excellent kappa statistics.
• Intraclass correlation coefficients for mean scores exceeded 0.70.
• Disagreement rates varied from 2% to 14%, with minimal reported difficulties despite participant preferences for certain modes.

Conclusions:

• The proposed four-step guide facilitates the evaluation of measurement comparability for PRO instruments under a crossover design.
• Investigating item-level responses, summary scores, and usability/preference provides comprehensive information for decision-making.
• This approach supports informed decisions on combining data across administration versions to simplify analysis and interpretation.