Weighting formulas for the least-squares analysis of binding phenomena data

Area of Science:

• Biophysics
• Biochemistry
• Analytical Chemistry

Background:

• Rectangular hyperbola models (y = abx/(1 + bx)) are common in complexation, sorption, fluorescence quenching, and enzyme kinetics.
• Directly measured independent variables (x) often have errors, violating least-squares assumptions and causing correlated errors in x and y.
• Linearized forms (reciprocal) of the hyperbola model are frequently used but can be problematic with data uncertainty.

Purpose of the Study:

• To develop weighting formulas for least-squares analysis of rectangular hyperbola data with errors in the independent variable (x).
• To evaluate the performance of different fitting methods, including reciprocal forms and the Deming-Lybanon algorithm.
• To provide guidance on appropriate data analysis techniques for common biochemical and biophysical models.

Main Methods:

• Utilized an effective variance approach to derive weighting formulas for the rectangular hyperbola and its linearized forms.
• Verified derived formulas by computing nonlinear least-squares parameter standard errors for exact data.
• Employed Monte Carlo simulations to assess the utility and limitations of different fitting methods.

Main Results:

• Derived weighting formulas for accurate least-squares analysis when the independent variable has errors.
• Monte Carlo simulations revealed that reciprocal fitting methods are unreliable for moderately uncertain data (approx. 30% uncertainty).
• The Deming-Lybanon algorithm consistently provides robust parameter estimates and standard errors, irrespective of the model's expression.

Conclusions:

• The effective variance approach provides a reliable method for handling errors in the independent variable for rectangular hyperbola models.
• Reciprocal forms of the model should be avoided when data uncertainty is significant.
• The Deming-Lybanon algorithm offers a superior and consistent approach for fitting such data, handling errors in multiple variables effectively.