Machine learning based variance estimation under two phase sampling using health and education sector data

This study proposes a novel variance estimator [Formula: see text]under two-phase sampling, utilizing one auxiliary variable and one binary attribute to enhance estimation efficiency. Theoretical properties of the estimator were obtained, such as the formula of bias and Mean Squared Error (MSE), which proves the analytical superiority of the estimator. The empirical efficiency of the simulation was demonstrated by the simulation performance in datasets of the health and education sectors, and the MSE values are consistently lower than those of the classical and competitive estimators. In further supporting its predictive power, machine learning classifiers (Regression Tree, Random Forest, and Support Vector Regression) were also trained on the same auxiliary inputs and evaluated benchmarked on the basis of Root Mean Squared Error (RMSE). Although Machine Learning (ML) models demonstrated good predictive power, the estimator used had good interpretability and theoretical foundations. Moreover, the three-parameter Weibull distribution was found to be the most suitable and cut-points were calculated to facilitate stratified interpretation of distributional analysis. In general, the research proves the usefulness of minimal auxiliary information as complemented by structured sampling and hybrid modeling to enhance precision of estimation of variance in applied studies.