Weighted least squares in calibration: estimating data variance functions in high-performance liquid chromatography

Area of Science:

• Analytical Chemistry
• Chromatography
• Statistical Modeling

Background:

• Accurate parameter estimation in analytical chemistry relies on appropriate weighting of heteroscedastic data.
• The method of least squares is commonly used, requiring data variance (sigma(i)^2) to inform weights (w(i) proportional to 1/sigma(i)^2).
• Understanding instrumental variance in high-performance liquid chromatography (HPLC) is crucial for reliable quantitative analysis.

Purpose of the Study:

• To estimate the instrumental data variance for a commercial HPLC instrument across a wide range of analyte concentrations and peak areas.
• To determine the functional form of the variance and identify its contributions.
• To evaluate the suitability of standard weighting schemes (1/x^2 or 1/y^2) for routine HPLC calibration and trace-level analysis.

Main Methods:

• Estimation of instrumental variance using 5 to 11 replicate measurements for over 20 samples across four analytes.
• Analysis of variance as a function of concentration and HPLC peak area, spanning over four orders of magnitude.
• Description of least-squares fitting methods for variance estimates, including direct and logarithmic fitting approaches.

Main Results:

• HPLC instrumental variance (s^2) is modeled as a sum of a constant term and a term proportional to the square of the peak area.
• The proportional error term, dominant in routine analysis, represents approximately 0.2% of peak area and includes a +/-0.008 microL injection volume uncertainty.
• Standard 1/x^2 or 1/y^2 weighting is justified for routine calibration but inaccurate at trace levels due to the constant variance component.
• Logarithmic fitting of variance estimates avoids iterative weight adjustment, unlike direct fitting.

Conclusions:

• Proportional error dominates HPLC instrumental variance in routine analysis, supporting inverse square weighting.
• A constant variance component necessitates adjustments to weighting schemes for accurate trace-level analysis.
• Logarithmic fitting offers a non-iterative approach for handling variance estimates with proportional uncertainty.