Predictive validity and interrater reliability of the FACE-CARAS toolkit in a CAMHS setting

The FACE-CARAS (Functional Analysis in Care Environments-Child and Adolescent Risk-Assessment Suite) toolkit has been developed to support practitioners in Child and Adolescent Mental Health Services (CAMHS) in performing a structured risk assessment. It covers a number of risk domains including violence, suicide, self-harm, experienced abuse, and exploitation. Interrater and internal reliability has already been established but not predictive validity.

Our aim was to establish the predictive validity of the FACE-CARAS in a CAMHS population.

Records from 123 young people with FACE-CARAS ratings completed by clinicians were examined in a retrospective file review to extract data on a relevant list of adverse outcomes at three and at 6 months following the assessment. Although this was not a prospective longitudinal study, researchers were blind to the clinicians' ratings, allowing valid testing of predictive power. Cases were drawn from across generic and specialist CAMHS teams in approximately equal proportions. Data were analysed using receiver operator characteristic statistics.

Areas under the curve values in five of the seven risk areas approached or were greater than 0.8 indicated that the FACE-CARAS profile score was a good potential predictor of risks of self-harm, suicidal behaviours, serious self-neglect, abuse or exploitation by others, and violence to others at both 3 and 6 months. It was weakly "predictive" of accidental self-harm and no better than chance at signalling physical ill health.

Findings support the use of the "profile summary" section of the tool as likely to generate clinically useful risk predictions. We were concerned that clinical use of the scale did not conform to research standards and often left subscales incompletely rated; however, the fact that the tool nonetheless proved a good predictor of most key adversities under scrutiny may add weight to its value in clinical practice. Further work with the FACE-CARAS subscales is recommended.