Multi-modal data fusion for enhanced pancreatic cancer detection

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Oncology

Background:

• Pancreatic cancer is a leading global cause of cancer death, often diagnosed late.
• Current computer-aided diagnosis (CADx) models predominantly use single data sources.
• Integrating diverse data types can potentially improve diagnostic performance.

Purpose of the Study:

• To investigate the effectiveness of various data-fusion strategies for multi-modal CADx in pancreatic cancer detection.
• To compare data-level, decision-level, and feature-level fusion methods.
• To evaluate performance on both a novel animal dataset and an internal pancreatic cancer dataset.

Main Methods:

• Developed and evaluated three data-fusion techniques: data-level, decision-level, and feature-level.
• Utilized a novel multi-modal animal dataset (2D images + attributes) and an internal pancreatic cancer dataset (3D CT scans + clinical features).
• Compared fusion methods against single-modality baselines (image-only and attribute-only).

Main Results:

• Multi-modal fusion significantly improved classification performance on both datasets.
• For pancreatic cancer detection, data-level and decision-level fusion outperformed single-modality approaches.
• The best model achieved an AUC of 0.94±0.01, surpassing image-only (0.87±0.01) and attribute-only (0.87±0.02) models.

Conclusions:

• Multi-modal CADx is highly effective for medical applications like pancreatic cancer detection.
• Advanced fusion methods hold promise for further improvements as more data becomes available.
• The study provides code and a new animal dataset to foster further research.