BAE-ViT: An Efficient Multimodal Vision Transformer for Bone Age Estimation.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Pediatric Radiology

Background:

• Traditional Convolutional Neural Networks (CNNs) lack the ability to efficiently merge image and non-image data for Bone Age Estimation (BAE).
• Integrating multimodal data, such as patient sex, is crucial for accurate BAE.

Purpose of the Study:

• To introduce BAE-ViT, a specialized vision transformer model for enhanced Bone Age Estimation (BAE).
• To develop a novel data fusion method for integrating visual and non-visual data in medical imaging.

Main Methods:

• Developed BAE-ViT, a vision transformer model for BAE.
• Implemented a novel data fusion technique by tokenizing non-visual data (e.g., sex) and concatenating it with image tokens.
• Trained the model on a large-scale dataset from the 2017 RSNA Pediatric Bone Age Machine Learning Challenge.

Main Results:

• BAE-ViT demonstrated commendable performance in Bone Age Estimation.
• The model showed superior performance in handling image distortions compared to existing models.
• Statistical analysis confirmed a strong correlation between BAE-ViT predictions and ground-truth labels.

Conclusions:

• Vision transformers offer a viable approach for integrating multimodal data in medical imaging.
• The proposed tokenization method enables effective incorporation of non-visual elements like sex into BAE.
• BAE-ViT provides a versatile framework for multimodal data integration in medical AI applications.