Related Experiment Video
Updated: Mar 23, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Personalized federated learning for medical vision-language models via efficient fine-tuning and uncertainty-aware
He Zhu1, Ren Togo2, Takahiro Ogawa2
1Graduate School of Information Science and Technology, Hokkaido University, Japan.
Abstract:
Privacy concerns and data heterogeneity remain critical barriers to advancing medical AI. While federated learning (FL) offers a decentralized solution, it often suffers from the Subspace Mismatch problem in clinical settings, where non-independent and identically distributed (non-IID) data drives clients towards divergent parameter manifolds. To address this, we propose a novel personalized federated learning (pFL) framework for vision-language medical tasks. Our approach employs a "separate-and-calibrate" strategy: First, we leverage Parameter-Efficient Fine-Tuning (PEFT) with a Dual-Subspace Disentanglement mechanism, splitting parameters into shared and personalized components to decouple general medical knowledge from site-specific characteristics. Second, to prevent negative transfer, we introduce Dynamic Likelihood-weighted Uncertainty Calibration (DLUC). This mechanism utilizes Dempster-Shafer theory to quantify epistemic uncertainty, acting as a reliability-aware gate that selectively aggregates compatible knowledge. Extensive experiments on Visual Question Answering, Question Generation, and Medical Report Generation demonstrate that our method significantly mitigates client drift and achieves superior personalization compared to state-of-the-art baselines.
Related Concept Videos
Improving Translational Accuracy
Improving Translational Accuracy
