Assessing the Robustness of Deep Learning Based Brain Age Prediction Models Across Multiple EEG Datasets

The increasing availability of large electroencephalography (EEG) datasets enhances the potential clinical utility of deep learning (DL) for cognitive and pathological decoding. However, dataset shifts due to variations in the population and acquisition hardware can considerably degrade the model performance. We systematically investigated the generalisation of DL models to unseen datasets with different characteristics, using age as the target variable. Five datasets were used in two different experimental setups, including leave-one-dataset-out (LODO) and leave-one-dataset-in (LODI) cross validation. 1805 different hyperparameter configurations were tested, with variations in the DL architectures and data pre-processing. The performance varied across source/target dataset pair. Using LODO, we obtained Pearson's r values of {0.63, 0.84, 0.75, 0.23, 0.10} and $R^{2}$ values of {-0.01, 0.63, 0.41, -4.66, -70.98}. For LODI, the results varied in Pearson's r from -0.11 to 0.84 and $R^{2}$ values from -704.89 to 0.65, depending on the source and target dataset. Our results show that DL models can learn age-related EEG patterns which generalise with strong correlations to datasets with broad age spans. The most important hyperparameter was to use the frequency range between 1 and 45 Hz, rather than a single frequency band. The second most important hyperparameter effect depended on the experimental setup. Our findings highlight the challenges of dataset shifts in EEG-based DL models and establish a benchmark for future studies aiming to improve the robustness of DL models across diverse datasets.