FULLY AUTOMATED DEEP LEARNING PIPELINE FOR HAND OSTEOARTHRITIS CLASSIFICATION REPLICATING THE OARSI ATLAS GRADING...

While the OARSI atlas grading system is useful for studies of hand OA, it is time consuming, expensive, and requires an expert reader.

To validate a fully automated Deep Learning (DL) pipeline that detects hands, standardizes hand orientation, detects radiographic R/L markers, locates and crops relevant joints, and performs OARSI JSN and OST grading classification at both binary (grade 0 or > 0) and multiclass (grades 0-3) levels.

Using 6810 OAI PA dominant hand radiographs (3405 participants) with available OARSI scoring, we developed a fully automated pipeline composed of five sequential stages. First, YOLOv8n object detection models were trained to detect radiographic R/L laterality markers hand orientation. During hand orientation training, marker regions were masked to prevent shortcut learning. Second, detected markers were assigned to the nearest hand based on bounding box centroid distance to determine anatomical laterality. Third, for joint localization, the HRNet-w32 model was trained on a manually annotated dataset of 987 images, including hand bounding boxes and 19 clinically relevant keypoints corresponding to the CMC, MCP, PIP, DIP, and thumb IP joints across the five digits. Fourth, data from stages 1-3 were used to generate individual joint-level crops. Fifth, DIP, PIP, and MCP joint crops from digits 2-5 with available OARSI labels were selected for JSN and OST classification. OARSI grade distributions were: JSN, grade 0: N=65,444, grade 1: N=12,687, grade 2: N=2,315, grade 3: N=1,219; OST, grade 0: N=62,891, grade 1: N=16,993, grade 2: N=1,539, grade 3: N=542. These crops were then used to train a ConvNeXt-Tiny model for binary and multiclass OARSI-based classification. Hand and marker detection were assessed using mAP50 and macro F1-score, and keypoint localization performance was evaluated using successful detection rate (SDR), as summarized in Figure 1. Final OARSI JSN and OST classification performance was evaluated using the macro F1-score, with detailed metrics reported in Table I. We also measured inter-reader reliability on DIP and PIP joints for a subset of radiographs from 70 participants not in the training set that were evaluated by Readers 1 (IKH) and 2 (SMB) and the DL model. The Pearson's r-value from a least-squares fit comparing the sums across all multiclass scores was used. Reader 1 data were used to train the models.

On the test sets, marker detection achieved an mAP50 of 0.98 and a macro F1-score of 0.97, while laterality detection achieved 0.99 for both metrics. Keypoint localization achieved an SDR@2mm of 0.95. For OARSI classification, macro F1-scores were 0.88 and 0.89 for binary JSN and OST models, respectively, and 0.78 and 0.72 for multiclass JSN and OST models, respectively. The reader reliability results were as follows: Reader 1 vs. Reader 2: r=0.89, Reader 1 vs. DL: r=0.94, Reader 2 vs. DL: r=0.88. These results suggest that the DL model is a valid surrogate for a human reader.

We have documented a well-performing fully automated DL-based software pipeline to assess hand radiographs for OARSI score that could be used to replace a human reader for clinical trials and studies of HOA. Future work will include DL-based segmentation to produce quantitative measures.