Raptor: Scalable Train-Free Embeddings for 3D Medical Volumes Leveraging Pretrained 2D Foundation Models.

Area of Science:

• Medical Imaging
• Deep Learning
• Computer Vision

Background:

• Developing foundational models for volumetric data (e.g., MRI) faces challenges in computational complexity and dataset curation.
• High-dimensional nature of volumetric data necessitates advanced computational techniques for effective analysis.

Purpose of the Study:

• Introduce Raptor (Random Planar Tensor Reduction), a train-free method for generating semantically rich embeddings for volumetric data.
• Address the computational and data challenges in developing deep learning models for medical volumes.

Main Methods:

• Leverage a frozen 2D foundation model pretrained on natural images to extract visual tokens from medical volume cross-sections.
• Employ random projections for spatial compression of tokens, reducing computational complexity while preserving semantic information.
• Utilize a train-free approach, eliminating the need for costly model training.

Main Results:

• Raptor demonstrates superior performance across ten diverse medical volume tasks compared to state-of-the-art methods.
• Achieved performance gains of +3% (SuPreM) to +14% (SLIViT) over existing methods.
• Validated effectiveness on various medical imaging datasets, highlighting versatility.

Conclusions:

• Raptor offers an effective and computationally efficient foundation for deep learning on medical volumes.
• The train-free methodology presents a significant advancement, reducing barriers to entry for volumetric data analysis.
• Highlights the potential of leveraging 2D models and random projections for complex 3D data tasks.