MTID-TS: Multimodal Training with Incomplete Data using Teacher-Student-Based Strategy on Medical Domain

Multimodal learning is a crucial technique in artificial intelligence (AI) that enhances model performance by integrating heterogeneous data modalities. However, real-world applications frequently suffer from uncertain missing modality conditions, particularly in medical and human activity recognition domain applications, which can degrade model reliability. To address these issues, we proposed a novel framework named Multimodal Training with Incomplete Data using a Teacher-Student-based Strategy (MTID-TS). This framework is based on knowledge distillation, where teacher models trained from the complete (single or dual) modality convey boosting information to the student model by transferring the probability output, thereby enhancing the student model performance under incomplete modality conditions. Besides, the proposed Missing Indicator Vector (MIV) indicates the modality availability to guide the model in adjusting its training and inference process. Furthermore, we integrate a stacking ensemble approach to enhance classification accuracy and employ GradNorm to adaptively balance the contributions of classification and distillation loss. We conducted extensive experiments on two real multimodal datasets to evaluate the generalization of our proposed framework in the medical and human activity recognition domain. Experimental results demonstrate that our approach improves classification performance and robustness under uncertain missing modality problems, outperforming existing methods. The proposed framework provides robust training and inference solutions for multimodal learning to adapt to real-world applications.