DermaGPT a federated multimodal framework with a meta learned trust function for interpretable dermatology...

Area of Science:

• Artificial Intelligence in Medicine
• Dermatology AI
• Federated Learning

Background:

• Generative and federated AI advance privacy-aware diagnostic systems.
• Multimodal reasoning and explainability are key for trustworthy AI in healthcare.

Purpose of the Study:

• Introduce DermaGPT, a federated multimodal framework for dermatology decision support.
• Emphasize trustworthy use with heterogeneous, privacy-sensitive data.

Main Methods:

• Combined PaLI-Gemma 2 vision-language backbone with retrieval-augmented LLM.
• Utilized meta-learned trust function (MLTF) for robust federated training.
• Evaluated on 4,452 biopsy-confirmed images across multiple datasets.

Main Results:

• Achieved 90.2% diagnostic accuracy for 11 lesion types.
• Reached 93.3% accuracy in malignancy prediction with well-calibrated outputs.
• Expert dermatologists found explanations clear and clinically relevant.

Conclusions:

• Trust-aware, federated multimodal design enables interpretable, efficient, and privacy-aware dermatology AI.
• DermaGPT augments, rather than replaces, clinician judgment.
• Local image processing and secure text transmission enhance privacy.