Reproducible research by design: design principles for low-code/no-code AutoML tools

Reproducibility is a major challenge for responsible machine learning (ML) research, especially with low-code/no-code automated ML (LCNC AutoML) tools making ML methods accessible to researchers outside data science. Existing approaches to support research reproducibility are difficult to apply to LCNC AutoML due to a lack of shareable code, limited tool transparency, and varying levels of ML expertise among users. To inform the design of LCNC AutoML tools that support reproducible research, this design science study proposes a set of design principles utilizing the theoretical lens of Technology Affordances and Constraints Theory. We first identified design requirements based on user studies with six researchers who implemented and reproduced LCNC AutoML projects. This was followed by a technical review of eight LCNC AutoML tools to identify existing design features in relation to the identified design requirements. Through mapping design features to design requirements, a set of design principles emerged: (1) GUI Customizability, (2) Standardized Reporting, (3) Interoperability, and (4) Adaptive Guidance. Together, these principles provide actionable design guidance for developing LCNC AutoML tools that support the documentation and reproducibility of ML research.