Quality assessment for systematic reviews and meta-analyses of cohort studies

There is a growing need for valid, efficient, and easy scoring scales to rate the quality of cohort studies. We aimed to develop and validate a quality assessment score to be used for cohort studies.

We followed a rigorous process to establish content, face, and construct validity. Most questions were scored at 0 or 1. Inter-rater reliability and test-retest reliability were assessed using the Spearman correlation coefficient (r_s) and Cohen's κ statistic. Internal consistency was measured using the Kuder-Richardson formula 20 (KR20).

The final tool consists of 9 questions with a maximum score of 10. The inter-rater reliability was high with the Spearman correlation coefficient (r_s = .66). Agreement for inclusion was 90%. Test-retest reliability was high. For rater 1, r_s = .91, κ = .38 for scores, and κ = 1 for inclusion. For rater 2, r = .94, 80% agreement for scores, and 100% agreement for inclusion. Internal consistency was reasonable based on 2 studies: KR20 = .21 and KR20 = .65. The novel scale rated highest in efficiency, understandably, ease of use, and ease of interpretation when compared with 3 other scales.

This novel scale has favorable performance characteristics, is efficient to conduct, and is easy to interpret and will be very helpful for physicians and researchers conducting systematic reviews and meta-analyses.