[Research on prediction of fracture reduction fixator therapy based on multimodal multi-label method]

To construct a prediction model for fracture reduction fixator therapy using the multi-modal multi-label classification (MMC) method.

Medical record data of 818 orthopedic patients from 2019 to 2023 were collected. Medical image features were extracted using the VGG19 network, text features of TCM four diagnostic methods (Inspection, Auscultation & Olfaction, Inquiry, Palpation) were extracted via the MiniLM model, and clinical case features were extracted through a fully connected neural network. After fusing the multi-modal information, multi-label therapy prediction was achieved using a linear layer.

Experimental results on the clinical multi-modal dataset showed that the MMC method performed excellently in terms of subset accuracy(SA), accuracy(Acc), precision, and F₁-score, reaching 0.661, 0.856, 0.897, and 0.899 respectively. When the image modality and text modality were removed, the model performance decreased by an average of 8.1% and 2.4% respectively, while the hamming loss(HL) increased by 21.1% and 5.6% respectively.

The fracture reduction fixator therapy prediction model constructed in this study can effectively fuse multi-modal data, accurately predict personalized treatment plans for patients, and significantly improve the accuracy and reliability of treatment decisions. It provides a new solution for the digitalization and intellectualization of Traditional Chinese Medicine(TCM) in fracture treatment and has important clinical application prospects.