MPLDM：人工股関節全置換術のためのマルチモーダル人工関節緩み診断モデル

Total hip arthroplasty (THA) is an effective procedure for restoring hip joint function and typically yields satisfactory clinical outcomes. Aseptic loosening and periprosthetic joint infection (PJI) are severe complications following total hip arthroplasty. Accurate diagnosis of these complications requires the integration of various clinical data, including X-ray images, CT scans, medical records, and laboratory test results. To efficiently utilize these critical data and robustly diagnose postoperative complications, we present the Multi-modal Prosthetic Loosening Diagnostic Model (MPLDM). MPLDM employs four independent encoders to convert the four types of data into unified feature tokens. To enhance the interaction between relevant modalities, we introduce three types of attention mechanisms for focusing on the important information. Finally, a multi-modal fusion module combines the information from these modalities to produce a robust prediction. Notably, our model is designed to generalize well even when some input modalities are missing, accommodating patients who cannot provide certain types of data. To effectively train and evaluate the proposed model, we propose a multi-modal dataset for diagnosing prosthetic loosening after THA. This dataset includes multi-modal input data for all four modalities, along with diagnostic results from senior clinical doctors. The MPLDM showed excellent performance with mean precision, recall and F1-score of 0.9140, 0.8353 and 0.8645, respectively. Experiments demonstrate that, compared to single-modal and other multi-modal baselines, our method shows superior accuracy and robustness in diagnosing both aseptic loosening and PJI-induced loosening.