TC-Net: A joint learning framework based on CNN and vision transformer for multi-lesion medical images segmentation

Area of Science:

• Medical Imaging
• Computer Vision
• Artificial Intelligence

Background:

• Accurate medical image segmentation is crucial for diagnosis and treatment.
• Existing methods struggle to integrate local and long-range information effectively.
• Advancements in medical imaging technology increase the demand for sophisticated segmentation tools.

Purpose of the Study:

• To develop an advanced segmentation framework for medical images.
• To effectively capture both locality-aware and long-range dependencies within medical images.
• To address the class imbalance issue in multi-lesion segmentation tasks.

Main Methods:

• Proposed TC-Net framework combining CNN-based encoder-decoder for local features and Transformer for global context.
• Introduced a locality-aware and long-range dependency concatenation strategy (LLCS) for feature aggregation.
• Developed a dynamic cyclical focal loss (DCFL) to handle class imbalance in multi-lesion segmentation.

Main Results:

• TC-Net achieved mean pixel accuracy of 0.6985 (IDRiD) and 0.5171 (DDR) on fundus image databases.
• Achieved a mean pixel accuracy of 0.8886 on a skin image database.
• Outperformed other deep learning segmentation schemes and demonstrated superior performance of DCFL over other loss functions.

Conclusions:

• TC-Net offers a promising solution for multi-lesion medical image segmentation.
• The framework's ability to integrate diverse dependencies enhances segmentation performance.
• TC-Net shows potential for various challenging image segmentation applications.