Psychometric properties of the DSM-5 Clinician-Rated Dimensions of Psychosis Symptom Severity

The Clinician-Rated Dimensions of Psychosis Symptom Severity (CRDPSS) was presented in the DSM-5 as a new scale to assess the dimensional aspects of psychosis in daily clinical practice. However, agreement in CRDPSS-ratings among raters in clinical practice remains unknown. We examined the inter-rater reliability (IRR) and convergent validity of the CRDPSS.

Consecutively recruited outpatients with recent onset schizophrenia spectrum disorders were included between January 2015 and July 2018. We collected multiple CRDPSS measurements of 335 participants, of whom 179 PANSS measurements were available. IRR was determined by comparing the CRDPSS-ratings of psychiatrists with a vis-à-vis contact and CRDPSS observations based on a detailed clinical presentation. IRR was expressed in Krippendorff's alpha and we estimated convergent validity by studying associations with PANSS factors by Spearman's rank correlation coefficient.

Inter-rater reliability scores measured in Krippendorff's alpha were low (0.35-0.64) for all items of the CRDPSS, except the item delusions (0.74). A three-factor model was found: 'deficit/motor symptoms', 'positive symptoms' and 'mood symptoms'. Positive associations between CRDPSS factors with PANSS factors were found.

This study demonstrated that the IRR of the CRDPSS between raters in clinical practice was insufficient. We did find some supporting evidence for convergent validity of the CRDPSS, but these results should be interpreted carefully due to low IRR. Consequently, general implementation in clinical practice should be done with caution and we recommend assessors to be trained.