Detection of Osteoarthritis from Multimodal Hand Data

Area of Science:

• Biomedical Engineering
• Medical Imaging
• Artificial Intelligence

Background:

• Osteoarthritis (OA) is a common degenerative joint disease affecting older adults, causing joint inflammation and swelling.
• Early and accurate detection of OA is crucial for effective management and treatment.

Purpose of the Study:

• To develop and evaluate a novel method for detecting hand osteoarthritis (OA) using a combination of imaging, video, and thermal data.
• To assess the performance of machine learning models in classifying OA in different hand joints.

Main Methods:

• Hand pose estimation from video data to calculate joint angles and create feature vectors.
• Training hand keypoint detectors on RGB and thermal images to extract joint-specific features.
• Utilizing Support Vector Machines (SVM) and Convolutional Neural Networks (CNN) for binary classification of OA in joints.

Main Results:

• The proposed method achieved favorable accuracy and F1-scores for Proximal Interphalangeal (PIP) and Distal Interphalangeal (DIP) joints.
• Performance on Metacarpophalangeal (MCP) joints was limited due to dataset imbalance.
• Video data, when used alone, yielded the best results among the individual modalities, outperforming combined approaches in some cases.

Conclusions:

• The developed method shows promise for utilizing visual and thermal data with machine learning for OA detection in hands.
• Further research and larger datasets are needed to improve performance on all joint types, particularly MCP joints.