Related Experiment Videos
Robustness and reproducibility for AI learning in biomedical sciences: RENOIR.
Alessandro Barberis1,2, Hugo J W L Aerts3,4,5,6, Francesca M Buffa7,8,9
1Nuffield Department of Surgical Sciences, Medical Sciences Division, University of Oxford, Old Road Campus Research Building, Roosevelt Drive, Oxford, OX3 7DQ, UK. dr.alessandro.barberis@gmail.com.
Scientific Reports
|January 22, 2024
Summary
RENOIR is an open-source platform enhancing artificial intelligence (AI) and machine learning (ML) reproducibility. It standardizes analysis pipelines, reports findings transparently, and improves model generalization, reducing research waste.
Area of Science:
- Computer Science
- Bioinformatics
- Pharmacology
Background:
- Artificial intelligence (AI) and machine learning (ML) are increasingly used with large datasets.
- However, AI publications often lack reproducibility and generalizability, leading to research waste.
- Standardized tools are needed to improve the reliability of AI/ML studies.
Purpose of the Study:
- Introduce RENOIR, a modular open-source platform for robust and reproducible machine learning analysis.
- Enhance the quality, transparency, and generalizability of AI studies.
- Address challenges in AI research, including sample size dependency and reporting.
Main Methods:
- RENOIR utilizes standardized pipelines for model training and testing.
- It incorporates novel elements like analyzing algorithm performance based on sample size.
- Automated generation of transparent and usable reports is a key feature.
Main Results:
- RENOIR was applied to benchmark datasets across health, computer science, and STEM domains.
- The platform successfully identified classifiers for SET2D and TP53 mutation status in cancer.
- A use case demonstrated RENOIR's utility in predicting drug efficacy for a pharmacological challenge.
Conclusions:
- RENOIR offers a solution to improve reproducibility and generalizability in AI/ML research.
- The platform's versatility is demonstrated across diverse scientific domains.
- RENOIR aims to reduce global research waste by enhancing the scientific value of AI studies.