NiCLIP: neuroimaging contrastive language-image pretraining model for predicting text from brain activation images

Predicting cognitive processes from brain activation maps remains a key challenge in neuroscience. Meta-analytic functional decoding provides an estimation of behavioral profiles associated with specific brain regions; however, existing methods fail to capture the full semantic content of the neuroimaging literature. To address this, we present NiCLIP, a contrastive language-image model trained on over 23,000 articles to predict cognitive tasks, concepts, and domains from brain activation patterns. NiCLIP achieves optimal performance when using full-text articles and a curated cognitive ontology with precise task-concept-domain mappings. Our results show that NiCLIP accurately predicts cognitive tasks from Human Connectome Project group-level maps across multiple domains (e.g., emotion, language, motor) and precisely characterizes the functional roles of specific brain regions, including the amygdala, hippocampus, and temporoparietal junction. However, NiCLIP showed limitations with noisy subject-level activation maps. NiCLIP represents a significant advancement in functional decoding, offering a powerful tool for hypothesis generation and scientific discovery.