Reliability of dermatologists in acne lesion counts and global assessments

There is a paucity of data on the reliability of dermatologists in acne lesion counting and global severity assessments. The effects of training and practice on reliability are also uncertain. The objective of this study was to determine the reliability of these outcome measurements when performed by trained dermatologists.

Eleven dermatologists were divided into two groups that evaluated the same six acne subjects twice on the same day. A training session was provided either after (group A) or before (group B) the first patient evaluation sessions. Reliability of raters in lesion counting and global severity assessment was determined by calculation of intraclass correlation coefficients (ICCs). ICC values close to 1.0 indicate excellent reliability, whereas values less than 0.75 are considered unacceptable.

Intrarater ICCs ranged from 0.37 to 0.99 for noninflammatory lesions, 0.26 to 0.97 for inflammatory lesions, and 0.56 to 0.83 for global assessments for group A (trained after); corresponding values for group B (trained before) were 0.84 to 0.98, 0.61 to 0.95, and 0.43 to 0.91. ICC values >or= 0.75 for all three outcome parameters were observed in one of six group A and three of five group B raters. Interrater ICCs for groups A and B after the first evaluation session were 0.17 versus 0.68 for noninflammatory counts, 0.84 versus 0.72 for inflammatory counts, and 0.71 versus 0.65 for global assessments, respectively. Corresponding values after session 2 were 0.79 and 0.77 for noninflammatory, 0.81 and 0.90 for inflammatory, and 0.61 and 0.77 for global assessments.

Dermatologists tended to be reliable in acne lesion counting but somewhat less so in global assessments. Training tended to improve group reliability in noninflammatory lesion counts and increased the proportion of raters with good reliability in all three outcome measures. Practice enhanced reliability in all outcome measurements.