Knockout: A simple way to handle missing inputs

Area of Science:

• Machine Learning
• Artificial Intelligence
• Data Science

Background:

• Deep learning models excel with multimodal inputs but face deployment challenges due to potential missing data during inference.
• Existing solutions like marginalization, imputation, and training multiple models have limitations, including computational cost, prediction inaccuracy, and the need for prior pattern knowledge.

Purpose of the Study:

• To propose an efficient and effective method for training deep learning models that can handle missing multimodal inputs during inference.
• To develop a technique that learns both conditional and marginal input distributions without requiring prior knowledge of missing data patterns.

Main Methods:

• Introduced 'Knockout,' a novel training strategy that randomly replaces input features with placeholder values.
• Provided theoretical justification for Knockout, demonstrating its interpretation as an implicit marginalization technique.
• Evaluated Knockout's performance across diverse simulations and real-world datasets.

Main Results:

• Knockout demonstrates strong empirical performance in handling missing multimodal data.
• The method efficiently learns both conditional and marginal distributions, offering a viable alternative to existing approaches.
• Knockout avoids the computational expense of marginalization and the potential inaccuracies of imputation.

Conclusions:

• Knockout presents an efficient and effective solution for deploying multimodal deep learning models with missing inputs.
• The method offers a robust alternative to traditional techniques, improving model generalizability and reducing deployment costs.
• Further research can explore the application of Knockout in various domains requiring robust multimodal data handling.