MIDAS: Mutual Information Disentanglement With Uncertainty-Aware Fusion for Incomplete Multimodal Sentiment Analysis.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Natural Language Processing

Background:

• Multimodal sentiment analysis often requires complete data, which is rare in real-world applications.
• Existing methods struggle to leverage incomplete multimodal data effectively due to reliance on imputation or heuristic constraints.

Purpose of the Study:

• To propose a unified framework, MIDAS, for robust multimodal sentiment analysis under incomplete data conditions.
• To effectively extract and leverage task-relevant information from incomplete multimodal datasets.

Main Methods:

• Developed a variational modeling strategy using multivariate Gaussian latent variables to represent modalities.
• Decomposed modal representations into shared and exclusive factors for disentanglement.
• Employed a minimax objective to stabilize disentanglement and enhance cross-modal semantic alignment.
• Introduced an uncertainty-aware fusion mechanism leveraging posterior variance as a reliability indicator.

Main Results:

• MIDAS demonstrated significant and consistent performance improvements across various incomplete data scenarios.
• The framework showed robustness in handling corrupted or missing modalities.
• Achieved superior results compared to competitive baselines on three benchmark datasets.

Conclusions:

• MIDAS offers an effective and robust solution for multimodal sentiment analysis with incomplete data.
• The proposed uncertainty-aware fusion mechanism enhances the integration of incomplete multimodal information.
• The framework's ability to restructure representations under missing modalities is key to its success.