Cross Attention Transformers for Multi-modal Unsupervised Whole-Body PET Anomaly Detection

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Oncology

Background:

• Cancer exhibits heterogeneous uptake patterns in positron emission tomography (PET) scans, crucial for diagnosis and staging.
• Unsupervised anomaly detection models can identify cancers by learning healthy tissue representations and detecting deviations.
• Transformers are well-suited for learning complex interactions in medical imaging for anomaly detection.

Purpose of the Study:

• To develop a multi-modal transformer model for enhanced cancer detection in PET scans.
• To improve cancer localization by integrating anatomical information from computed tomography (CT) scans via cross-attention.
• To evaluate the model's robustness and performance, especially when healthy training data is scarce.

Main Methods:

• Utilized a transformer architecture with multi-modal conditioning through cross-attention, integrating PET and CT imaging data.
• Trained the model on 83 whole-body PET/CT scans encompassing diverse cancer types.
• Employed model uncertainty and kernel density estimation for robust anomaly mapping, serving as an alternative to residual-based methods.

Main Results:

• The proposed model demonstrated robust and accurate cancer localization capabilities, even in the absence of healthy training data.
• The uncertainty quantification provided a statistically reliable method for generating anomaly maps.
• Achieved superior performance compared to leading alternative anomaly detection approaches.

Conclusions:

• Multi-modal transformer models with cross-attention significantly enhance unsupervised anomaly detection for cancer in PET/CT imaging.
• The approach offers a robust solution for cancer localization, particularly valuable when healthy data is limited.
• This method shows promise for improving cancer diagnosis, staging, and treatment monitoring through advanced AI in medical imaging.