ViruSeg: Harnessing the Power of Large Language-Image Model for Enhanced Virus Image Segmentation

Area of Science:

• Virology
• Medical Imaging
• Artificial Intelligence

Background:

• Novel viral diseases like SARS-CoV-2 present significant global health challenges.
• Accurate viral identification and segmentation are vital for disease tracking and treatment development.
• Existing segmentation models face limitations due to scarce labeled data, morphological variability, and unclear boundaries.

Purpose of the Study:

• To introduce ViruSeg, an efficient virus segmentation model.
• To address challenges of limited data, morphological variability, and indistinct boundaries in viral imaging.
• To enhance the accuracy and adaptability of virus segmentation for research and treatment.

Main Methods:

• Employed data augmentation techniques (cutout, image fine-tuning) to enrich electron microscope images.
• Utilized the EVA-02 large language-image pre-trained model for universal virus image representation.
• Performed virus segmentation using the Cascade Mask R-CNN (CMR) model.

Main Results:

• ViruSeg demonstrated superior performance in virus segmentation compared to existing advanced methods.
• The model effectively delineates virus boundaries and different forms, improving generalization.
• Enhanced adaptability to data scarcity was achieved through the EVA-02 pre-trained model.

Conclusions:

• ViruSeg offers a robust solution for accurate viral image segmentation.
• The proposed method advances virology research and aids in developing treatments for viral diseases.
• The study provides open-access datasets and code for further research and development.