A dual alignment-based multi-source domain adaptation framework for motor imagery EEG classification

Domain adaptation, as an important branch of transfer learning, can be applied to cope with data insufficiency and high subject variabilities in motor imagery electroencephalogram (MI-EEG) based brain-computer interfaces. The existing methods generally focus on aligning data and feature distribution; however, aligning each source domain with the informative samples of the target domain and seeking the most appropriate source domains to enhance the classification effect has not been considered. In this paper, we propose a dual alignment-based multi-source domain adaptation framework, denoted DAMSDAF. Based on continuous wavelet transform, all channels of MI-EEG signals are converted respectively and the generated time-frequency spectrum images are stitched to construct multi-source domains and target domain. Then, the informative samples close to the decision boundary are found in the target domain by using entropy, and they are employed to align and reassign each source domain with normalized mutual information. Furthermore, a multi-branch deep network (MBDN) is designed, and the maximum mean discrepancy is embedded in each branch to realign the specific feature distribution. Each branch is separately trained by an aligned source domain, and all the single branch transfer accuracies are arranged in descending order and utilized for weighted prediction of MBDN. Therefore, the most suitable number of source domains with top weights can be automatically determined. Extensive experiments are conducted based on 3 public MI-EEG datasets. DAMSDAF achieves the classification accuracies of 92.56%, 69.45% and 89.57%, and the statistical analysis is performed by the kappa value and t-test. Experimental results show that DAMSDAF significantly improves the transfer effects compared to the present methods, indicating that dual alignment can sufficiently use the different weighted samples and even source domains at different levels as well as realizing optimal selection of multi-source domains.