Masked image modeling-based boundary reconstruction for 3D medical image segmentation.

Area of Science:

• Medical image analysis
• Computer vision
• Artificial intelligence

Background:

• Accurate 3D medical image segmentation is crucial for computer-aided diagnosis.
• Challenges include morphological variations, limited labeled data, and integrating global/local information.
• Existing models struggle with efficient information integration for structured knowledge acquisition.

Purpose of the Study:

• To introduce a novel transformer-based 3D model, the TNT Masking Network (TNT-MNet), for improved medical image segmentation.
• To address challenges in data scarcity and information integration using masked image modeling (MIM) in supervised learning.
• To enhance structured knowledge acquisition by utilizing target boundary regions as masked prediction targets.

Main Methods:

• Developed the TNT Masking Network (TNT-MNet) featuring a transformer-in-transformer (TNT) encoder.
• Implemented masked image modeling (MIM) in supervised learning, masking target boundary regions for prediction.
• Employed multiscale random masking on inner and outer tokens in an online branch, with a target branch guiding reconstruction.

Main Results:

• TNT-MNet demonstrates performance comparable or superior to state-of-the-art models.
• Evaluated on three medical image datasets: BTCV, LiTS2017, and BraTS2020.
• Significantly reduces the dependence on labeled data for medical image segmentation tasks.

Conclusions:

• TNT-MNet offers a groundbreaking approach to 3D medical image segmentation.
• The model effectively integrates global and local information, enhancing structured knowledge acquisition.
• The proposed MIM strategy in supervised learning shows promise for data-efficient segmentation.