Discovering hierarchical common brain networks via multimodal deep belief network

Studying a common architecture reflecting both brain's structural and functional organizations across individuals and populations in a hierarchical way has been of significant interest in the brain mapping field. Recently, deep learning models exhibited ability in extracting meaningful hierarchical structures from brain imaging data, e.g., fMRI and DTI. However, deep learning models have been rarely used to explore the relation between brain structure and function yet. In this paper, we proposed a novel multimodal deep believe network (DBN) model to discover and quantitatively represent the hierarchical organizations of common and consistent brain networks from both fMRI and DTI data. A prominent characteristic of DBN is that it is capable of extracting meaningful features from complex neuroimaging data with a hierarchical manner. With our proposed DBN model, three hierarchical layers with hundreds of common and consistent brain networks across individual brains are successfully constructed through learning a large dimension of representative features from fMRI/DTI data.