MANNERS: A strategy for representation learning in multivariate datasets with high proportions of missing data

Area of Science:

• Machine Learning
• Data Science
• Biomedical Informatics

Background:

• Missing data is a significant challenge in deep learning, especially in electronic health records where missing rates can exceed 99%.
• Traditional imputation methods struggle with high missingness, potentially losing valuable information present in missingness patterns.

Purpose of the Study:

• To develop a novel strategy, MANNERS, to effectively handle missing data in deep learning for representation learning.
• To leverage patterns within missing data, rather than just imputing it, to improve model performance.

Main Methods:

• MANNERS (Missing Adjusted Normalization and Nullity Encoding Representation Strategy) encodes missingness and masks loss calculations at missing points.
• It employs rebalancing techniques to ensure that variables with high missing rates contribute meaningfully to the learning process.

Main Results:

• MANNERS demonstrated improved performance in reconstruction, classification, regression, and synthetic data generation tasks.
• Significant performance gains were observed specifically in scenarios with very high missing data rates compared to imputation-only methods.

Conclusions:

• MANNERS offers a robust approach to deep learning with high-dimensional, incomplete time-series data, particularly from electronic health records.
• This strategy effectively utilizes missingness information, enhancing representation learning and downstream task performance beyond state-of-the-art imputation techniques.