Distributed variational bayesian meta-learning over multi-agent systems

Learning new tasks from limited data inherently introduces uncertainty in meta-learning. While Bayesian meta-learning aims to enhance traditional algorithms by quantifying uncertainty in model parameters, existing methods are centralized and cannot handle decentralized data. In this paper, we present the Distributed Variational Bayesian Meta-Learning (DVBML) framework, the first to enable distributed uncertainty quantification for decentralized meta-learning. Unlike deterministic approaches like Dif-MAML, DVBML models meta-parameters as probability distributions. We propose a novel gradient-diffusion hybrid algorithm that combines local stochastic gradient variational Bayes steps with a global diffusion-based consensus strategy, allowing agents to collaboratively learn and propagate both the mean and variance of task parameters. Rigorous theoretical analysis proves that this strategy guarantees convergence and achieves a linear speedup O(1/NT) in variance reduction compared to non-cooperative baselines. Extensive evaluations on few-shot regression and classification benchmarks show that DVBML matches the accuracy of centralized Bayesian meta-learning and delivers reliable, well-calibrated uncertainty estimates. It significantly outperforms non-cooperative baselines and quantitatively surpasses deterministic distributed methods in calibration metrics, maintaining remarkable robustness even under imbalanced data settings.