Non-linear regression analysis with errors in both variables: estimation of co-operative binding parameters

Four different parameter estimation criteria, the geometric mean functional relationship (GMFR), the maximum likelihood (ML), the perpendicular least-squares (PLS) and the non-linear weighted least squares (WLS), were used to fit a model to the observed data when both regression variables were subject to error. Performances of these criteria were evaluated by fitting the co-operative drug-protein binding Hill model on simulated data containing errors in both variables. Six types of data were simulated with known variances. Comparison of the criteria was done by evaluating the bias, the relative standard deviation (S.D.) and the root-mean-squared error (RMSE), between estimated and true parameter values. Results show that (1) for data with correlated errors, all criteria perform poorly; in particular, the GMFR and ML criteria. For data with uncorrelated errors, all criteria perform equally well with regard to the RMSE. (2) Use of GMFR and ML lead to lower values for S.D. but higher biases compared with WLS and PLS. (3) WLS performs less well when equal dispersion is applied to the two observed variables.