The Bland-Altman method should not be used in regression cross-validation studies

Area of Science:

• Biostatistics
• Sports Science
• Exercise Physiology

Background:

• Regression models are frequently used to estimate physiological parameters like maximum oxygen uptake.
• The Bland-Altman (BA) method is a common tool for assessing agreement between measurement methods.
• The appropriateness of the BA method for validating regression models requires careful examination.

Purpose of the Study:

• To demonstrate the inherent bias of the Bland-Altman (BA) limits of agreement method when applied to the validation of regression models.
• To highlight the limitations of the BA method in assessing the accuracy of predictive equations.
• To advocate for alternative, more appropriate methods for regression model validation.

Main Methods:

• Development of three regression equations to estimate maximum oxygen uptake using data from 1,158 men.
• Cross-validation of these regression models using a separate sample of 581 men.
• Application and analysis of the Bland-Altman (BA) method to assess agreement and identify bias in the cross-validation sample.

Main Results:

• Significant correlations between BA means and differences were observed for all three models (r = .55, .39, and .26, p < .001), indicating systematic bias.
• The BA method demonstrated a clear bias, confirming its unsuitability for validating regression models.
• The degree of bias varied across models with different explanatory power (R2 values of .40, .61, and .82).

Conclusions:

• The Bland-Altman (BA) method is inappropriate for the validation of regression models due to inherent biases.
• Accurate validation of regression equations should involve plotting residuals against estimated values.
• Assessing the magnitude of estimation error through residual analysis is the recommended approach for regression model validation.