TrainTracks - federated learning for reproducible research on sensitive medical data

Area of Science:

• Medical AI
• Computational Biology
• Data Science

Background:

• Reproducibility of computational algorithms is critical for trustworthy AI in medical research.
• Federated Learning (FL) enables privacy-preserving AI but requires robust traceability for reproducibility.
• Existing traceable FL platforms often rely on blockchain, necessitating resource-efficient alternatives for healthcare.

Purpose of the Study:

• To propose and evaluate TrainTracks, a novel concept for reproducible and traceable federated learning in medical research.
• To extend the Personal Health Train (PHT) Platform for Analytics and Distributed Machine Learning for Enterprises (PADME) with enhanced traceability features.
• To integrate privacy-preserving change tracing for data, metadata, and experiment execution using DataLad and MetaLad.

Main Methods:

• Extended the PADME platform to incorporate TrainTracks for reproducible and traceable FL.
• Integrated DataLad and MetaLad for privacy-preserving change tracing of data, metadata, and computational experiments.
• Evaluated TrainTracks against a detailed checklist for reproducible AI requirements.

Main Results:

• TrainTracks improved PADME's compliance with reproducible AI guidelines by 15 points out of 47 applicable to FL.
• Significant improvements were observed in data reproducibility, with full support for automatic information extraction in 10 of 12 points.
• Method reproducibility saw no direct improvements, but experiment reproducibility was enhanced through workflow and code traceability.

Conclusions:

• Combining FL with data versioning tools, as in TrainTracks, creates an automated workflow for tracing algorithms and data.
• TrainTracks demonstrates high compliance with recommendations for reproducible AI experiments, methods, and data.
• Complete FL process traceability, including dynamic dataset versioning, is essential for reproducible medical research.