DIFFERENTIAL DIAGNOSIS OF HAND ARTHRITIS ON RADIOGRAPHS USING DEEP LEARNING

Radiographic differential diagnosis of hand arthritides remains challenging, particularly when findings overlap across rheumatoid arthritis (RA), psoriatic arthritis (PsA), and hand osteoarthritis (HOA), a common situation in older patients. Deep learning may support image-based classification, but clinical datasets are often limited, imbalanced, and affected by cohort-related heterogeneity.

To present initial work-in-progress results of a deep learning-based four-class classification approach distinguishing RA, PsA, HOA, and no documented radiographic changes (NDRC) on hand radiographs from the Medical University of Vienna (MUW) and the Osteoarthritis Initiative (OAI).

This retrospective feasibility study included 5,404 unilateral hand radiographs from MUW and OAI, split at the patient level into 4,814 training, 201 validation, and 389 independent test radiographs (Table 1). Target classes were rheumatoid arthritis (RA), psoriatic arthritis (PsA), hand osteoarthritis (HOA), and no documented radiographic changes (NDRC), with NDRC indicating absence of documented arthritic changes rather than clinically verified healthy controls. MUW and OAI cases were pooled for HOA and NDRC, while RA and PsA radiographs originated from MUW. Images were classified using an ImageNet-pretrained ResNet-18. Performance on the independent test set was assessed using accuracy, balanced accuracy, macro and weighted F1-scores, Cohen's kappa, and macro-averaged multiclass ROC-AUC.

In the mixed four-class setting, the model achieved an accuracy of 0.59, balanced accuracy of 0.59, macro F1-score of 0.58, weighted F1-score of 0.58, Cohen's kappa of 0.45, and macro-averaged multiclass ROC-AUC of 0.84. Per-class performance is shown in Table 2, with recall ranging from 0.38 to 0.79 and F1-scores from 0.41 to 0.80. The relatively high ROC-AUC compared with moderate accuracy and F1-scores suggests that the model captured discriminative ranking information, whereas final single-label four-class assignment remained challenging in this heterogeneous multi-source setting.

These initial findings suggest that deep learning can capture disease-related signal in heterogeneous hand radiograph datasets, but robust differential diagnosis of RA, PsA, and HOA, as well as separation from NDRC, remains challenging. A key limitation is that NDRC reflects absence of documented radiographic changes in the source data rather than clinically verified healthy controls. Future work will focus on expanding the dataset with more balanced multi-centre data, including radiographs from healthy individuals, and improving robustness to cohort-related domain shift.