Assessment of CDASI scoring by a multimodal large language model: a comparative study with expert assessors

Large language models (LLMs) with multimodal capabilities may support automated assessment of cutaneous disease activity in dermatomyositis (DM). We evaluated the performance of Claude v3.5 Sonnet in scoring the Cutaneous Dermatomyositis Disease Area and Severity Index (CDASI) compared with expert rheumatologists. Thirty published DM cases with available clinical images were retrospectively analyzed. Two expert rheumatologists independently scored CDASI activity and damage domains. Claude v3.5 Sonnet assessed the same images using structured prompting based on CDASI definitions. Agreement was evaluated using intraclass correlation coefficients (ICCs) with a two-way random-effects model for absolute agreement. The LLM demonstrated better agreement for activity than for damage assessment. Global CDASI activity showed good agreement with expert raters (ICC 0.71, 95% CI 0.45-0.86), whereas damage reliability was lower (ICC 0.41, 95% CI 0.25-0.67). Agreement was moderate for activity-related domains including erythema (ICC 0.61), scaling (ICC 0.57), and erosions (ICC 0.57), while lower concordance was observed for chronic damage-related features such as poikiloderma (ICC 0.47, 95% CI 0.10-0.73). Hand assessments demonstrated the strongest performance, particularly for periungual changes (ICC 1.0, 95% CI 1.0-1.0) and global hand scores (ICC 0.95, 95% CI 0.91-0.97). The model also markedly reduced evaluation time compared with clinicians (42 s vs. 8.4 min per case), corresponding to an approximately 92% reduction in scoring time. Multimodal LLMs demonstrated promising agreement with expert raters for CDASI activity assessment and markedly improved efficiency, although lower reliability in damage domains reinforces the continued need for physician oversight.