Does the Clinical Context Improve the Reliability of Rheumatologists Grading Digital Ulcers in Systemic Sclerosis?

Digital ulcers (DUs) are often a primary end point in systemic sclerosis (SSc; scleroderma) clinical trials, although the reliability of rheumatologists grading DUs is poor to moderate at best. DU assessment in recent trials has been based upon visual inspection alone, which potentially misses "real-world" clinical contextual information. Our aim was to investigate whether this clinical information improves the reliability of rheumatologists grading DUs. A secondary aim was to assess agreement between patients and rheumatologists.

Eighty images of a range of digital lesions were collected from patients with SSc with the clinical context: pain (severity and temporal relationship), lesion duration, and discharge (patient reported and clinician observed). Raters received all images either with or without the clinical context, and graded these images (using a custom-built interface) on an ordinal scale of severity: no ulcer, inactive ulcer, or active ulcer. Patients also graded their lesion(s) on the same scale.

Fifty-one rheumatologists from 15 countries completed the study (26 without and 25 with context): 4,590 (including 510 repeated) image gradings were obtained. Context did not significantly increase (without and with context) either intra- (0.64, 0.71) or interrater (0.32, 0.36) reliability. Pain (visual analog scale and temporal relationship) and discharge (patient reported and clinician observed) were associated with increased lesion severity, and duration with reduced severity. Agreement between individual patients and rheumatologists was poor without and with context (0.19, 0.28).

The overall intra- and interrater reliability of DU grading did not significantly improve with the clinical context. Agreement between patients and rheumatologists was poor.