Gradient Importance Learning for Incomplete Observations

Area of Science:

• Machine Learning
• Data Science
• Artificial Intelligence

Background:

• Traditional methods for handling missing data often rely on imputation, which can introduce errors and degrade performance in downstream tasks like classification.
• These imputation-based approaches struggle with datasets exhibiting high missingness rates or small sample sizes, and imputation errors can propagate and limit prediction models.
• Existing imputation techniques may not align with real-world data complexities, hindering the effectiveness of subsequent analyses.

Purpose of the Study:

• To introduce a novel imputation-free method for directly performing inference on data with missing values.
• To develop a technique that leverages missingness patterns to improve model training and prediction accuracy.
• To overcome the limitations of traditional two-step imputation-then-prediction methods.

Main Methods:

• Gradient Importance Learning (GIL) trains multilayer perceptrons (MLPs) and long short-term memories (LSTMs) to directly infer from inputs containing missing values.
• Reinforcement learning (RL) is employed to adjust back-propagation gradients, enabling models to learn from missingness patterns.
• The approach is designed to avoid the imputation step entirely, processing missing values directly within the model architecture.

Main Results:

• The GIL method demonstrated superior performance in imputation-free prediction tasks compared to traditional imputation-based methods.
• Evaluations on diverse datasets, including MIMIC-III time-series, eye clinic tabular data, and MNIST, confirmed the effectiveness of the proposed approach.
• Predictions generated without imputation outperformed those using state-of-the-art imputation techniques across tested datasets.

Conclusions:

• The proposed Gradient Importance Learning (GIL) method offers an effective imputation-free strategy for machine learning models dealing with missing data.
• This approach successfully exploits missingness patterns, leading to improved predictive performance and overcoming limitations of conventional imputation techniques.
• GIL provides a robust alternative for handling missing data in real-world applications, particularly for time-series and tabular datasets.