Combination of FMRI-SMRI-EEG data improves discrimination of schizophrenia patients by ensemble feature selection

Multimodal brain imaging data fusion is a scientifically interesting and clinically important topic; however, there is relatively little work on N-way data fusion. In this paper, we applied multi-set canonical correlation analysis (MCCA) to combine data of resting state fMRI, EEG and sMRI, in order to elucidate the abnormalities that underlie schizophrenia patients and also covary across multiple modalities. We also tested whether the identified group-discriminative components can be used for feature selection in group classification. MCCA is demonstrated to be an effective feature selection technique, especially in multimodal fusion. We also proposed an ensemble feature selection scheme by combining two sample t-test, MCCA and support vector machine with recursive feature elimination (SVM-RFE), resulting in optimal group-discriminating features for each modality. Finally, we compared the classifying power between two groups based on the above selected features via 7 modality-combinations. Results show that the fMRI-sMRI-EEG combination derives the best classification accuracy in training (91%) and predication rate (100%) in testing data, validating the effectiveness and advantages of multimodal fusion in discriminating schizophrenia.