Hierarchical Mixture-of-Experts Framework for Whole-Brain Imaging and Whole-Genome Analysis of Multiple Cognitive...

Understanding the biological mechanisms of different cognitive functions is essential for assessing brain health and developing proper interventions. Neuroimaging and genetic data provide complementary information for studying these mechanisms. However, it remains challenging to extract task-specific features from each modality and effectively integrate them for prediction because of the high-dimensional and heterogeneous multimodal data. To address this challenge, we propose IGMoE, a hierarchical mixture-of-experts framework, which jointly performs task-guided feature selection and multi modal fusion for whole-brain imaging and whole-genome analysis. First, we design two modality-specific sparse mixture-of-experts networks (SMoE) to extract, group, and select task-relevant features from imaging and genetic data separately. Contrastive pretraining is then employed to align multimodal representations. In addition, a multimodal fusion mixture-of-experts network (FMoE) is proposed to dynamically integrate the selected features to predict multiple cognitive scores. We evaluate IGMoE using MRI and genetic data from the UK Biobank (UKB) dataset on three cognitive tasks: paired associate learning, matrix pattern completion, and trail making, as well as from the ADNI and PPMI datasets on the trail making task. Experimental results show that IGMoE outperforms existing methods in both unimodal and multimodal data, demonstrating its effectiveness for multi-task whole-brain and whole-genome analysis.