Consistency of orthodontists' clinical decisions: A systematic review, meta-analysis, and theory development

The purpose of this systematic review and meta-analysis is to provide a literature-based estimate of the consistency of orthodontists' clinical decisions.

A systematic review of the literature using a modified Reporting Items for Systematic Reviews and Meta-Analyses approach identified 20 articles, representing 53 unique datasets, reporting kappa statistics and standard errors for situations allowing intrarater or interrater comparison on decisions such as the need for treatment, extraction, surgery, and various specific treatment approaches. Meta-regression based on random effect models was used to explore the shape of the underlying distribution, the prevalence of the target condition in the data set, and the professional experience of raters as covariables.

No evidence of publication bias was found. Common patient records accounted for approximately 25% of the variance between orthodontists and 33% of the variance within orthodontists making the same decision from the same records. Random and representative samples were judged more consistently than were samples chosen to contain borderline cases. (P <0.001). Raters were in greater agreement on the presence of target conditions than their absence (P <0.001). Residents were more consistent than were practicing orthodontists or dental students (P <0.006).

Low consistency was found among orthodontists making clinical decisions from common records. Factors associated with samples and raters suggest an underlying pattern of orthodontists viewing cases through personal mental frameworks.