Interpretable multimodal machine learning for diagnosis of drug-resistant tuberculosis

Early detection of drug-resistant tuberculosis (DR-TB) is critical for timely treatment and limiting transmission. Differentiating DR-TB from drug-sensitive TB (DS-TB) remains challenging due to overlapping manifestations and subtle differences in chest X-rays (CXRs).

We developed an explainable multimodal machine learning framework integrating CXR embeddings from a Data-Efficient Image Transformer (DEiT) with patient-level data, including clinical, interpreted radiological, demographic, socioeconomic, and behavioral variables. Using 6,279 subjects from the TB Portals program, model performance was evaluated across three settings comprising CXR embeddings only, patient-level features only, and multimodal. Interpretability was analyzed using Shapley Additive Explanations (SHAP).

A multimodal XGBoost model achieved a precision-recall area under the curve (PR-AUC) of 0.9258 and receiver operating characteristic area under the curve (ROC-AUC) of 0.8718. CXR-embeddings-only classifiers performed lower (PR-AUC: 0.8334; ROC-AUC: 0.7369), while patient-level XGBoost yielded highest performance. The multimodal model improved interpretability through complementary structural information. SHAP identified CXR embeddings as primary global drivers. Among patient-level variables, daily contacts and employment status were the strongest contributors, reflecting exposure intensity and socioeconomic conditions. Comorbidity showed moderate effects, while education and case definition had smaller contributions. Key radiological factors included non-tuberculosis abnormalities and mediastinal lymph node involvement, whereas medium-density stabalized fibrotic nodules, medium nodules, and small cavities demonstrated minimal association. Interaction analysis revealed higher risk with increasing comorbidity in the presence of morphological alterations; BMI and age exhibited modest influence. Independent review by two clinicians confirmed plausibility and consistency of SHAP explanations.

This interpretable multimodal framework supports early DR-TB detection and targeted interventions in high-burden settings.