systemPipeR: a multipurpose workflow management system for reproducible data analysis

Area of Science:

• Bioinformatics
• Computational Biology
• Genomics

Background:

• Workflow management systems (WMS) are crucial for reproducible multi-step data analyses.
• Existing WMS often lack deep integration with R and Bioconductor, key ecosystems for statistical and scientific analysis.
• A need exists for unified solutions linking R-based analyses with command-line tools.

Purpose of the Study:

• To present systemPipeR, an R-based workflow management system (WMS) that bridges R/Bioconductor with command-line software.
• To introduce a redesigned architecture for streamlined workflow construction, execution, monitoring, and reporting.
• To facilitate reproducible end-to-end data analysis in genomics and other data-intensive fields.

Main Methods:

• Developed a unified environment linking R-based analytical steps with command-line tools via standardized workflow specifications.
• Implemented a redesigned architecture featuring a flexible workflow management class object.
• Integrated the Common Workflow Language (CWL) for standardized execution of R and command-line steps, including metadata management and automated reporting.

Main Results:

• systemPipeR now offers a unified environment for designing and executing reproducible workflows on diverse computing systems.
• The new architecture simplifies workflow management, from construction to reporting.
• Enhanced integration with CWL and improved metadata handling facilitate complex data analysis.

Conclusions:

• systemPipeR provides a robust, general-purpose, R-based WMS for reproducible analysis of complex biological data.
• Its unified approach addresses the gap in integrating R/Bioconductor with command-line tools.
• The latest version offers significant improvements for efficient and reliable scientific workflows.