Towards a framework for interoperability and reproducibility of predictive models

Area of Science:

• Biomedical informatics
• Machine learning applications in healthcare
• Computational biology

Background:

• Standardized methodologies for developing and deploying machine learning (ML) models in biomedical research and healthcare are currently lacking.
• Existing tools for model replication do not provide a unifying blueprint, hindering scientific reproducibility due to unclear assumptions, preprocessing steps, and test metrics.
• Challenges in generalizability and transportability of ML models remain significant issues in the field.

Purpose of the Study:

• To facilitate scientific reproducibility in biomedical machine learning.
• To present the Automated Metadata Pipeline (AMP) as a key component of the PREdictive Model Index and Exchange REpository (PREMIERE) platform.
• To enable the conversion of predictive ML models into an extended PMML format for enhanced interoperability and reproducibility.

Main Methods:

• Development of the Automated Metadata Pipeline (AMP) building upon the Predictive Model Markup Language (PMML).
• Conversion of predictive ML models into extended PMML files.
• Autocompletion of an ML-based checklist to assess model elements for interoperability and reproducibility.
• Demonstration of the pipeline on multiple test cases using three different ML algorithms and health-related datasets.

Main Results:

• Successful conversion of ML models into extended PMML files using the AMP.
• Demonstrated assessment of model elements for interoperability and reproducibility via an autocompleted checklist.
• Validation of the pipeline across diverse ML algorithms and health datasets.

Conclusions:

• The Automated Metadata Pipeline (AMP) provides a foundational solution for enhancing the reproducibility of predictive ML models in healthcare.
• The extended PMML format facilitates better model sharing, comparison, and understanding of generalizability and transportability.
• This work paves the way for more robust and reliable ML applications in biomedical research and clinical practice.