The analysis and interpretation of method comparison studies in rehabilitation research

Rehabilitation researchers are frequently interested in comparing the results of two tests that measure the same function. There are practical as well as financial advantages to having more than one method of assessing a particular component of function. Investigations examining the results of two tests are referred to as method comparison studies and commonly analyzed using the Pearson product-moment correlation coefficient (r). Several researchers have argued that the Pearson r is a measure of linear association (co-variation) between variables and does not provide accurate estimates of direct agreement. We compared several commonly used quantitative methods to establish agreement and have demonstrated that the Pearson r is not appropriate for use in studies where the purpose is to determine whether two instruments are interchangeable. An alternative to the Pearson r for analyzing data from method comparison studies is presented. The new procedure, referred to as the limits of agreement method, is easy to compute and emphasizes the clinical comparability of two instruments (or raters) instead of focusing solely on the statistical relationship.