Alternative methods for evaluating the equivalence of measurement systems

Reports are common in the orthodontic and dental radiographic literature comparing measurement systems. Typically, such comparisons are made using differences in mean scores across methods, correlation coefficients, or Bland-Altman plots. These methods are subject to known limitations, including an inability to detect bias.

A hypothetical dataset was created to contain a small, common random variance and two types of bias to compare three alternative measurement systems against a common standard. One comparison included only random error and no bias. Two types of bias were investigated: systemic overestimation on the part of one measurement system and the more complex case of overestimation in one part of the range and underestimation in the other.

Each of the commonly used methods for comparing alternative measurement systems was inadequate for detecting one or the other type of bias. The traditional regression analysis, by contrast, provided a useful characterization of the alternative measurement systems, including quantification of the nature of bias.

Regression analysis can reveal biases masked in the common comparison of means, correlation coefficients, and Bland-Altman plots for evaluating alternative measurement systems and thereby improve confidence in clinical applicability of research.