Generative Neural Networks for Data Imputation in Longitudinal Epidemiological Studies

Area of Science:

• Epidemiology
• Machine Learning
• Biostatistics

Background:

• Longitudinal epidemiological studies frequently encounter incomplete follow-up and missing data, potentially biasing results and reducing statistical power.
• Conventional imputation methods struggle with the complex patterns and dependencies inherent in multivariate time series data, especially with irregular intervals and extensive missingness.
• Existing generative machine learning models offer improvements but often lack the capability to handle inconsistently spaced measurements and completely missing time steps common in long-term health outcome evaluations.

Purpose of the Study:

• To develop and evaluate a novel variational autoencoder-based generative neural network for imputing missing information in irregular time series data.
• To address the challenges of extensive and patterned missingness common in longitudinal epidemiological research.
• To provide a robust method for reconstructing partially and fully missing values in long-term health studies.

Main Methods:

• Implementation of a variational autoencoder (VAE)-based generative neural network.
• Exploitation of correlations between features at single time steps and temporal trends of features over time for value reconstruction.
• Testing on synthetic data mimicking longitudinal epidemiological study characteristics and a real-world dataset.

Main Results:

• Demonstrated effectiveness of the proposed VAE-based generative network in imputing missing data within irregular time series.
• Superior performance and parameter stability compared to prior methods across various degrees and patterns of missingness.
• Successful reconstruction of partially and fully missing information in both synthetic and real-world datasets.

Conclusions:

• The developed generative neural network provides an effective solution for imputing missing data in longitudinal epidemiological studies with irregular and extensive missingness.
• The approach shows promise for improving the accuracy and power of analyses in long-term health outcome research.
• This method offers a significant advancement over conventional imputation techniques for complex, real-world time series data.