Related Experiment Videos
T2ID: Towards trustworthy incomplete modality diagnosis via dual-axis adaptive completion and confidence-aware
Jing Li1, Qinkai Yu2, Feixiang Zhou3
1Guangdong Provincial Key Laboratory of Multimodal Big Data Intelligent Analysis, South China University of Technology, Guangzhou, China; Department of Computer Science, University of Exeter, Exeter, United Kingdom; Department of Eye and Vision Sciences, University of Liverpool, Liverpool, United Kingdom; Bioengineering Program, Biomedical Sciences Division (BioMed), King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia.
Abstract:
Multimodal medical images provide abundant and complementary diagnostic cues that are often indispensable for accurate clinical decision-making. However, various constraints, such as limited equipment availability, physical limitations of patients, etc. often lead to incomplete modality of medical data acquisition in the real world. This scenario makes it difficult to design robust and trustworthy diagnostic AI models under missing-modality conditions. In addition, for some samples, cross-modal completion may not always be beneficial when the available modality representation can already support a reliable prediction. Blindly completing every missing modality may introduce noisy or mismatched complementary features, increase unnecessary computation, and potentially degrade model performance. To address these limitations, we propose T2ID, a multimodal diagnostic framework, which adaptively completes missing modalities and explicitly models cross-modal interactions while maintaining trustworthiness and robustness against missing modality scenarios. Our framework incorporates two key innovations. Firstly, we design a Dual-axis Adaptive Modality Completion (DAA) module, which integrates vertical and horizontal completion strategies to efficiently estimate whether retrieval-based cross-modal completion is expected to provide reliable complementary evidence for the current sample, or cross-modal retrieval is needed for adaptive modality completion. Secondly, we introduce Dynamic Collaborative Confidence-aware Modality Integration (DCC) module, which estimates both unimodal-confidence and synergistic-confidence to capture synergistic or conflicting interactions across modalities, thereby ensuring trustworthy diagnostic predictions. Extensive experiments on three multimodal medical imaging datasets demonstrate that our T2ID achieves superior performance compared with other state-of-the-art methods under diverse missing-modality scenarios. The implementation code is publicly available at https://github.com/lijing-coder/T2ID.