FluoroSAM: A Language-promptable Foundation Model for Flexible X-ray Image Segmentation.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Existing medical image analysis models are often task-specific, limiting their broad applicability.
• Language-aligned foundation models (LFMs) show promise for automated image analysis but require extensive data.
• X-ray imaging presents unique challenges due to high variability in appearance and applications.

Purpose of the Study:

• To develop a language-promptable foundation model for comprehensive X-ray image analysis.
• To enable flexible human-in-the-loop workflows in diagnostic and interventional medicine.
• To address the data variability and limited annotations in X-ray imaging.

Main Methods:

• Introduced FluoroSAM, a variant of the Segment-Anything Model trained on 3 million synthetic X-ray images.
• Incorporated vector quantization (VQ) of text embeddings for natural language prompt integration.
• Utilized diverse anatomies, imaging geometries, and viewing angles with pseudo-ground truth masks for organs and tools.

Main Results:

• FluoroSAM successfully segments numerous anatomical structures and tools from X-ray images using natural language prompts.
• Demonstrated quantitative performance on real X-ray data.
• Showcased FluoroSAM's utility in various applications, facilitating human-machine interaction.

Conclusions:

• FluoroSAM advances language-promptable X-ray image segmentation.
• The model enhances flexibility for human-in-the-loop workflows in medical imaging.
• FluoroSAM is a key enabler for improved X-ray image acquisition and analysis.