Multimodal model for knee osteoarthritis KL grading from plain radiograph

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Orthopedics

Background:

• Knee osteoarthritis is a global health issue with no cure.
• Early identification is crucial for managing osteoarthritis progression.
• Manual X-ray interpretation for osteoarthritis grading is subjective and prone to errors.

Purpose of the Study:

• To develop a multimodal AI model for accurate knee osteoarthritis severity grading (Kellgren-Lawrence classification).
• To leverage pre-trained vision and language models for enhanced feature extraction from X-ray images and associated text data.
• To improve the objectivity and accuracy of osteoarthritis assessment compared to traditional methods.

Main Methods:

• A multimodal model integrating Vision Transformer (ViT) for image embeddings and BERT for text embeddings was developed.
• Transformer encoders were utilized to extract distinctive hidden states for the neural network classifier.
• The model was trained and validated using the Osteoarthritis Initiative (OAI) dataset.

Main Results:

• The multimodal model achieved high performance in identifying knee osteoarthritis severity.
• The model demonstrated an overall accuracy of 82.85% on the test set.
• Precision reached 84.54% and recall was 82.89% in experimental evaluations.

Conclusions:

• The proposed multimodal AI model shows significant potential for accurate knee osteoarthritis grading.
• This approach offers a more objective and reliable method for assessing disease severity compared to manual interpretation.
• The findings support the use of advanced AI in medical diagnostics for improved patient management.