SCImputation: Mitigating Feature Confounding From a Structural Causal Perspective for Data Imputation.

Area of Science:

• Data Science
• Biostatistics
• Machine Learning

Background:

• Missing data pose significant challenges in data analysis, often causing biased estimates and reduced reliability.
• Existing imputation methods frequently overlook the influence of the feature with the missing value, leading to suboptimal predictions.

Purpose of the Study:

• To propose a novel data imputation strategy, Structural Causal Imputation (SCImputation), grounded in structural causal models.
• To mitigate confounding introduced by the target feature during neighbor selection in imputation.

Main Methods:

• Adopted a structural causal perspective to analyze data imputation.
• Developed SCImputation to refine neighbor selection using both instance-level and feature-level information.
• Applied a back-door adjustment formula to reweight local estimates with global distribution, correcting for confounding.

Main Results:

• SCImputation variants achieved 3.0%-4.6% gains in accuracy and 0.009-0.059 reductions in Root Mean Square Error (RMSE) compared to 12 baselines.
• Demonstrated competitive performance against leading deep learning baselines across diverse missingness mechanisms.
• Evaluated on five diverse datasets including NACC and NCBI microarrays.

Conclusions:

• SCImputation offers a causally grounded strategy for addressing missing data challenges.
• The method provides significant improvements in accuracy and reliability for data imputation.
• Applicable to both biomedical and general data analysis scenarios.