Related Experiment Video
Updated: Jun 9, 2025
Author Spotlight: Advancing 3D Modeling for Enhanced Diagnosis and Treatment of Pulmonary Nodules in Early-Stage Lung Cancer
Published on: October 13, 2023
Foundational Segmentation Models and Clinical Data Mining Enable Accurate Computer Vision for Lung Cancer.
Nathaniel C Swinburne1, Christopher B Jackson2, Andrew M Pagano3
1Department of Radiology, Memorial Sloan Kettering Cancer Center, 1275 York Ave, New York, NY, 10065, USA. swinburn@mskcc.org.
This study shows that integrating Segment Anything Model (SAM) and MedSAM into automated mining, object detection, and segmentation (MODS) effectively detects and segments lung cancer in CT scans without manual labeling. The models achieve high performance, demonstrating their potential for radiology applications.
Area of Science:
- Medical Imaging
- Artificial Intelligence in Medicine
- Computational Pathology
Background:
- Automated lung cancer detection and segmentation in medical images is crucial for early diagnosis and treatment planning.
- Existing methods often require extensive manual annotation, which is time-consuming and labor-intensive.
- Foundational segmentation models offer a potential solution for reducing annotation burden.
Purpose of the Study:
- To evaluate the effectiveness of integrating the Segment Anything Model (SAM) and its variant MedSAM into the automated mining, object detection, and segmentation (MODS) methodology.
- To develop robust lung cancer detection and segmentation models using only mined clinical data, eliminating the need for post hoc labeling.
- To assess the performance of these models on both internal and external lung cancer datasets.
Main Methods:
- A retrospective analysis of 10,000 chest computed tomography (CT) scans from lung cancer patients was performed.
- Line measurement annotations were converted to bounding boxes for training the You Only Look Once (YOLO) object detection architecture.
- Teacher-student learning was employed for self-labeling, followed by training a final tumor detection model integrated with SAM and MedSAM for segmentation.
Main Results:
- The automated mining, object detection, and segmentation (MODS) methodology successfully generated 5403 training boxes from 10,789 line annotations.
- The baseline detection model achieved an internal F1 score of 0.847, improving to 0.860 after self-labeling.
- Tumor segmentation using SAM and MedSAM achieved internal Dice Similarity Coefficients (DSCs) of 0.842 and 0.822, respectively, with external validation showing comparable results after fine-tuning.
Conclusions:
- Integrating foundational segmentation models like SAM and MedSAM into the MODS framework yields high-performing lung cancer detection and segmentation models.
- This approach effectively utilizes mined clinical data, significantly reducing the reliance on manual post hoc labeling.
- Both SAM and MedSAM demonstrate considerable promise as foundational segmentation models for processing radiology images.
