Comparisons of different classification algorithms while using text mining to screen psychiatric inpatients with...

To compare the performance of methods based on text mining to screen suicidal behaviors according to chief complaint of the psychiatric inpatients.

Electronic Medical Records of inpatients with mental disorders were collected. Text mining method was adopted to screen suicidal behaviors. The performances of different combinations of six algorithms and two term weighting factors were compared under various training set sizes, which were assessed by precision, recall, F1-value and accuracy.

A total of 3600 psychiatric inpatients (1800 with suicidal behaviors and 1800 without suicidal behaviors) were included in this study. In chief complaints of suicidal inpatients, "suicide", "notion" and "suspicion" were the commonest statements, appearing 1228, 705 and 638 times respectively. In contrast, "excitement", "instability" and "impulsion" appeared more frequently in chief complaints of patients without suicidal behaviors (599, 599, 534 times respectively). The performance of each algorithm was generally improved with the increasing training set sizes and tended to be stable when the number of training cases reached 1000, where most of them could achieve satisfactory accuracy values (>0.95). Results of testing set showed that SVM, Random Forest and AdaBoost weighted by TF had better generalization ability. The F1 values were 0.9889 for SVM, 0.9838 for random forest and 0.9828 for AdaBoost, respectively.

This study confirmed the feasibility of filtering suicidal inpatients with small amounts of representative terms. SVM, Random Forest and AdaBoost weighted by TF have better performance in this task. Our findings provided a practical way to automatically classify patients with or without suicidal behaviors before admission to hospital, which potentially led to considerable savings in time and human resources for identification of high-risk patients and suicide prevention.