Examiner alignment and assessment in clinical periodontal research

Periodontal research typically relies on clinical examiners to assess variables such as gingival inflammation, plaque scores or probing depths as a means of determining treatment outcomes or for performing group comparisons. The quality of the gathered information is dependent, to a large extent, on the skills of the examiner(s) and on the validity of the assessment methods that are used. Attempts have been made to increase the objectivity of periodontal assessments, for example by introducing scoring systems for gingival inflammation, but within these systems there is often considerable scope for variation when interpreting the scoring criteria, leading to subjectivity when assigning scores to individual periodontal sites. This has led to an awareness of the importance of examiner alignment and assessment to improve the data quality by standardizing techniques and improving examiner reliability. Examiner alignment and assessment is used in preference to the term 'examiner calibration' because calibration implies comparison with an accurate or 'gold' standard, which is not available in periodontal research. In this review, we consider the historical perspective that led to the development of clinical scoring systems for periodontal research using gingival inflammation as an example. A clinical protocol for undertaking examiner alignment and assessment is presented, and we review the common sources of error and bias that can lead to difficulties in aligning examiners, and consider how they can be eliminated. It is particularly important that subjects who are recruited to the examiner alignment and assessment study present with a comparable level of disease to the subjects who will ultimately be recruited to the planned clinical trial. Another challenge in examiner alignment and assessment is applying appropriate statistical tests to assess the outcome of the alignment exercise. In the periodontal literature, the statistic kappa is frequently used to confirm an adequate degree of examiner agreement, but kappa is bound to significant restrictions when applied for this purpose. Through the use of case studies, we present different approaches to data analysis from calibration studies, focusing on continuous variables (such as probing depths and attachment levels) or ordinal data (such as gingival or plaque indices), to enable a correct, although frequently conservative, interpretation of data generated during examiner alignment and assessment studies.