Multiple Instance Neuroimage Transformer

Area of Science:

• Neuroimaging
• Artificial Intelligence
• Medical Image Analysis

Background:

• Transformer models are increasingly used in neuroimaging.
• Convolutional neural networks (CNNs) have limitations in capturing long-range dependencies in 3D neuroimages.
• Multiple instance learning (MIL) offers a framework for handling complex data structures.

Purpose of the Study:

• To propose and evaluate the Multiple Instance Neuroimage Transformer (MINiT), a novel convolution-free transformer model for T1-weighted MRI classification.
• To adapt transformer architectures for neuroimage analysis, focusing on a multiple instance learning approach.
• To demonstrate the model's capability in identifying sex-based differences in brain morphometry.

Main Methods:

• Developed MINiT, a multiple instance learning-based, convolution-free transformer model.
• Processed T1-weighted MRIs by dividing them into non-overlapping 3D blocks and further into 3D patches.
• Applied multi-headed self-attention to these patches for feature extraction and classification.
• Trained the model on the Adolescent Brain Cognitive Development (ABCD) and National Consortium on Alcohol and Neurodevelopment in Adolescence (NCANDA) datasets to predict sex.

Main Results:

• The MINiT model successfully classified sex from T1-weighted MRIs.
• Learned attention maps identified specific brain regions and voxels associated with sex differences in morphometry.
• Demonstrated the efficacy of a transformer-based MIL approach for neuroimage analysis.

Conclusions:

• MINiT represents a novel and effective deep learning approach for neuroimage classification.
• The model's attention mechanisms provide insights into neuroanatomical correlates of sex differences.
• This work highlights the potential of transformer architectures in advancing medical image analysis.