Development of a large-scale grounded vision language dataset for chest CT analysis

Area of Science:

• Artificial Intelligence in Medicine
• Medical Imaging Analysis
• Foundation Models

Background:

• Developing generalist foundation models in AI for Medicine requires diverse, open-source medical image datasets.
• Existing datasets often lack comprehensive supervision signals across various imaging modalities.

Purpose of the Study:

• Introduce RadGenome-Chest CT, a novel, large-scale, region-guided 3D chest CT interpretation dataset.
• Enhance AI for Medicine by providing a dataset suitable for multimodal foundation model development.

Main Methods:

• Leveraged universal segmentation models and large language models to extend existing datasets.
• Incorporated organ-level segmentation masks for 197 categories.
• Generated 665K multigranularity grounded reports and 1.2M grounded visual question answering (VQA) pairs.

Main Results:

• Created a dataset with extensive region-guided annotations for 3D chest CT interpretation.
• Enabled linking of textual reports and VQA pairs to specific anatomical regions via segmentation masks.
• Facilitated training for text generation based on segmented regions.

Conclusions:

• RadGenome-Chest CT significantly advances the development of multimodal medical foundation models.
• The dataset's unique features enable training capabilities not achievable with previous datasets.
• Poised to accelerate progress in AI-driven medical image interpretation.