Calibration and reliability testing of a novel asynchronous photographic plaque scoring system in young children

The Simplified Oral Hygiene Index for Maxillary Incisors (OHI-MIS) is a novel plaque scoring system adapted for young children. This study describes calibration training and testing used to establish the inter- and intra-rater reliability for OHI-MIS measured from clinical photographs.

Two raters from the Coordinated Oral Health Promotion Chicago (CO-OP) and one from the Behavioral EConomics for Oral health iNnovation (BEECON) randomized controlled trials (RCTs) underwent calibration with gold standard raters, followed by annual re-calibration. Raters from CO-OP also completed inter-rater reliability testing; all three raters completed intra-rater reliability testing rounds. Photographs were obtained from children aged 9-39 months.

All three raters achieved greater than 0.77 Lin's Concordance Correlation (LCC) versus gold standard consensus during calibration. All three raters had LCC ≥0.83 at recalibration 1 year later. CO-OP trial raters scored 604 photos (151 sets of 4 photographs); mostly both raters were somewhat/very confident in their scoring (≥89%), describing the most photos as "clear" (90% and 81%). The CO-OP inter-rater LCC for total OHI-MIS score was 0.86, changing little when low quality or confidence photos were removed. All three raters demonstrated high intra-rater reliability (≥0.83).

The OHI-MIS plaque scoring system on photos had good reliability within and between trials following protocol training and calibration. OHI-MIS provides a novel asynchronous plaque scoring system for use in young children. Non-clinicians in field or clinical settings can obtain photographs, offering new opportunities for research and clinical care.