Color painting predicts clinical symptoms in chronic schizophrenia patients via deep learning

Individuals with psychiatric disorders perceive the world differently. Previous studies indicated impaired color vision and weakened color discrimination ability in psychotic patients. Examining the paintings from psychotic patients can measure the visual-motor function. However, few studies examined the potential changes in the color painting behavior in these individuals. The current study aims to discriminate schizophrenia patients from healthy controls (HCs) and predict PANSS scores of schizophrenia patients according to their paintings.

In the present study, we retrospectively analyzed the paintings colored by 281 chronic schizophrenia patients and 35 HCs. The images were scanned and processed using series of computational analyses.

The results showed that schizophrenia patients tend to use less color and exhibit different strokes compared to HCs. Using a deep learning residual neural network (ResNet), we were able to discriminate patients from HCs with over 90% accuracy. Further, we developed a novel convolutional neural network to predict PANSS positive, negative, general psychopathology, and total scores. The Root Mean Square Error (RMSE) of the prediction was low, which indicates higher accuracy of prediction.

In conclusion, the deep learning paradigm showed the large potential to discriminate schizophrenia patients from HCs based on color paintings. Besides, this color painting-based paradigm can effectively predict clinical symptom severity for chronic schizophrenia patients. The color paintings by schizophrenia patients show potential as a tool for clinical diagnosis and prognosis. These findings show potential as a tool for clinical diagnosis and prognosis among schizophrenia patients.