Meta-analysis prompt guided multi-task graph transformer for joint analysis of cognitive deficits and psychiatric...

Psychiatric disorders, such as schizophrenia and attention-deficit/hyperactivity disorder (ADHD), are associated with altered functional connectivity (FC) and are often accompanied by cognitive deficits. Leveraging the shared neural mechanisms underlying both psychiatric disorders and cognitive impairment may improve diagnostic accuracy. However, due to the complexity and heterogeneity of these conditions, diagnosis based on FC alone remains challenging in terms of accuracy and biomarker reliability. To address these challenges, we propose a meta-analysis prompt guided multi-task graph transformer network that simultaneously predicts psychiatric disorder and cognitive deficits while identifying associated alterations in brain FC. The proposed framework employs a graph transformer as a common encoder and integrates a joint attention mechanism to capture shared disorder-cognition representations, together with saliency pooling to identify task-relevant brain regions. To enhance the robustness and interpretability of the learned saliency patterns, we introduce large language models (LLMs) to encode meta-analysis priors derived from the BrainMap database. Specifically, activation probability maps constructed from 466 functional neuroimaging studies are converted into natural-language prompts and embedded using LLMs, providing semantically rich and flexible priors that guide feature learning beyond handcrafted correlation constraints. In addition, subject-level demographic information is encoded as LLM-based text embeddings and incorporated into the model to support individualized prediction. A contrastive learning objective is further designed to align text-based embeddings with FC-derived representations. Experiments on the COBRE, ADHD-200, and SRPBS datasets, comprising 842 subjects in total, demonstrate that our method outperforms six single-task methods and seven state-of-the-art multi-task learning approaches in both schizophrenia and ADHD classification, as well as in the prediction of related cognitive deficits. The identified biomarkers are consistent with established findings while also revealing discriminative features, underscoring their potential relevance for future treatment development and longitudinal follow-up studies.