Models Genesis | JoVE Visualize

Area of Science:

Medical Imaging
Deep Learning
Computer Vision

Background:

Transfer learning from natural images to medical images is a common deep learning paradigm.
Current 3D medical imaging tasks are often reformulated in 2D, losing crucial 3D anatomical context and compromising performance.
Existing 3D models and 2D transfer learning methods have limitations in fully capturing 3D anatomical information.

Purpose of the Study:

To develop a novel deep learning framework for 3D medical image analysis that overcomes the limitations of 2D reformulation.
To introduce Generic Autodidactic Models (Models Genesis) that learn from self-supervision without manual labeling.
To demonstrate the superiority of Models Genesis over existing methods in 3D medical image segmentation and classification.

Main Methods:

Developed a unified self-supervised learning framework for 3D medical imaging.
Created Generic Autodidactic Models (Models Genesis) trained ex nihilo and by self-supervision.
Utilized the inherent anatomical structure in medical images as self-supervision signals.
Evaluated Models Genesis on five diverse 3D medical imaging applications (segmentation and classification).

Main Results:

Models Genesis significantly outperformed models trained from scratch and existing pre-trained 3D models across all tested applications.
Models Genesis consistently surpassed all 2D/2.5D approaches, including fine-tuning ImageNet pre-trained models and 2D versions of Models Genesis.
The study confirmed the critical importance of 3D anatomical information for superior performance in medical image analysis.

Conclusions:

Models Genesis represent a significant advancement in 3D medical image analysis by effectively utilizing 3D anatomical information through self-supervision.
The self-supervised learning framework provides a powerful and generalizable approach for creating source models for various 3D medical imaging tasks.
The open availability of codes and pre-trained models facilitates further research and application in the field.