Analysis of Missingness Scenarios for Observational Health Data

Area of Science:

• Health Informatics
• Statistical Methodology
• Observational Data Analysis

Background:

• Extensive literature exists on missing data theory, yet a gap remains in integrating domain knowledge into missing data methods for healthcare.
• Realistic analysis of healthcare data requires addressing how data becomes missing and its implications.

Purpose of the Study:

• To propose a framework for identifying key missingness scenarios in healthcare data.
• To investigate the theoretical implications of these scenarios on missing data analysis steps.
• To enhance the reliability of statistical methods through domain-informed analysis.

Main Methods:

• Developed an analysis framework to assess how observation agents (e.g., physicians) influence data availability.
• Applied the framework to observational healthcare data, identifying ten fundamental missingness scenarios.
• Investigated the impact of these scenarios on graphical models, inverse probability weighting, and sensitivity analysis.

Main Results:

• Identified ten fundamental missingness scenarios influencing missing data graphical models, inverse probability weighting, and exponential tilting.
• Simulation studies demonstrated that domain-informed analysis improves method reliability for variable mean estimation and classification accuracy.
• Compared complete-case analysis, MissForest imputation, and inverse probability weighting under various missingness scenarios.

Conclusions:

• Advocating for the proposed analysis framework as a reference for observational health data analysis.
• The framework is applicable to various medical domains beyond the initial case study.
• Integrating domain knowledge is essential for robust missing data handling in healthcare.