Distributed Weight Consolidation: A Brain Segmentation Case Study.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Neuroimaging

Background:

• Training deep neural networks requires large datasets, which are difficult to collect due to data sharing restrictions.
• Existing continual learning methods for distributed data require sequential training, limiting efficiency.
• Derivative datasets or predictive models may be shared with fewer restrictions than raw data.

Purpose of the Study:

• To introduce Distributed Weight Consolidation (DWC), a novel continual learning method for consolidating neural networks trained on independent datasets.
• To evaluate DWC's effectiveness in a brain segmentation task using structural magnetic resonance imaging (sMRI) data from multiple sites.

Main Methods:

• Developed Distributed Weight Consolidation (DWC) to combine weights of separate neural networks trained on independent datasets.
• Applied DWC to consolidate dilated convolutional neural networks trained on distributed sMRI datasets.
• Compared DWC performance against an ensemble baseline.

Main Results:

• DWC demonstrated increased performance on test sets from individual sites.
• The method maintained generalization performance on a large, independent multi-site dataset.
• DWC outperformed the ensemble baseline in the brain segmentation case study.

Conclusions:

• Distributed Weight Consolidation (DWC) offers an effective approach for continual learning with distributed datasets.
• DWC facilitates model improvement by combining insights from independent data sources without sequential training.
• The method shows promise for applications like multi-site neuroimaging analysis where data sharing is challenging.