A general method for describing sources of variance in clinical trials, especially operator variance, in order to...

The purpose of this study was to demonstrate how the skill level of the operator and the clinical challenge provided by the patient affect the outcomes of clinical research in ways that may have hidden influences on the applicability of that research to practice. Rigorous research designs that control or eliminate operator or patient factors as sources of variance achieve improved statistical significance for study hypotheses. These procedures, however, mask sources of variance that influence the applicability of the conclusions. There are summary data that can be added to reports of clinical trials to permit potential users of the findings to identify the most important sources of variation and to predict the likely outcomes of adopting products and procedures reported in the literature.

Provisional crowns were constructed in a laboratory setting in a fully crossed, random-factor model with two levels of material (Treatment), two skill levels of students (Operator), and restorations of two levels of difficulty (Patient). The levels of the Treatment, Operator, and Patient factors used in the study were chosen to ensure that the findings from the study could be transferred to practice settings in a predictable fashion. The provisional crowns were scored independently by two raters using the criteria for technique courses in the school where the research was conducted.

The Operator variable accounted for 38% of the variance, followed by Treatment-by-Operator interaction (17%), Treatment (17%), and other factors and their combinations in smaller amounts. Regression equations were calculated for each Treatment material that can be used to predict outcomes in various potential transfer applications. It was found that classical analyses for differences between materials (the Treatment variable) would yield inconsistent results under various sampling systems within the parameters of the study.

Operator and Treatment-by-Operator interactions appear to be significant and previously underrecognized sources of variance. It is suggested that variance estimates of factors thought to significantly influence the transfer of research findings to practice contexts and evidence of representative sampling across practice contexts be regularly included in reports of clinical trials.