Multimodal AI for Home Wound Patient Referral Decisions From Images With Specialist Annotations

Area of Science:

• Artificial Intelligence
• Medical Informatics
• Computer Vision

Background:

• Chronic wounds impact millions, particularly the elderly and diabetic populations.
• Home care for chronic wounds involves nurses with variable expertise, leading to inconsistent referral decisions.
• Inaccurate or delayed referrals for non-healing wounds can result in severe adverse outcomes, including limb amputation.

Purpose of the Study:

• To develop and evaluate a novel machine learning framework, the Deep Multimodal Wound Assessment Tool (DM-WAT), for assisting visiting nurses in making wound referral decisions.
• To leverage smartphone-captured wound images and clinical notes for automated wound assessment.
• To improve the accuracy and timeliness of referrals for chronic wound management.

Main Methods:

• DM-WAT utilizes a Vision Transformer (ViT) architecture (DeiT-Base-Distilled) for visual feature extraction from wound images.
• Text features are extracted from clinical notes using DeBERTa-base, capturing contextual information.
• Visual and text features are fused using an intermediate approach, enhanced by data augmentation and transfer learning to address small, imbalanced datasets.

Main Results:

• DM-WAT achieved a 77% accuracy and a 70% F1 score in rigorous evaluations.
• The proposed framework outperformed existing state-of-the-art, single-modality, and multimodal approaches.
• Interpretation algorithms (Score-CAM, Captum) provided insights into the model's decision-making process, highlighting key image and text features.

Conclusions:

• The Deep Multimodal Wound Assessment Tool (DM-WAT) demonstrates significant potential in supporting visiting nurses with objective wound referral recommendations.
• This AI-powered tool can enhance the management of chronic wounds, potentially reducing adverse events and improving patient outcomes.
• The multimodal approach, combined with advanced deep learning techniques, offers a robust solution for wound assessment in non-clinical settings.