Watchdog - a workflow management system for the distributed analysis of large-scale experimental data

Area of Science:

• Bioinformatics
• Computational Biology
• Genomics

Background:

• High-throughput experimental technologies generate large, complex datasets requiring sophisticated analysis.
• Bioinformatic analysis involves multiple, interdependent steps across numerous samples and conditions.
• Workflow management systems (WMSs) are crucial for automating and ensuring reproducibility of these analyses.

Purpose of the Study:

• To introduce Watchdog, a novel WMS for automated analysis of large-scale experimental data.
• To provide a flexible and user-friendly platform for constructing and executing bioinformatic workflows.
• To facilitate efficient handling of replicate data and support distributed computing environments.

Main Methods:

• Watchdog is implemented in Java, ensuring platform independence.
• Features include straightforward replicate data processing, distributed system support, and customizable error handling.
• A graphical user interface (GUI) aids workflow construction with pre-defined modules, complemented by a command-line and web interface for execution and monitoring.

Main Results:

• Watchdog enables automated analysis of large-scale experimental data with features for reproducibility and reusability.
• The system supports distributed computing and offers customizable error detection with manual intervention capabilities.
• Example workflows for RNA-seq analysis are provided, demonstrating the system's utility on real-life data.

Conclusions:

• Watchdog is a powerful and flexible WMS for high-throughput data analysis.
• It benefits users of all programming skill levels, simplifying the development and application of bioinformatic workflows.
• The software, documentation, and example workflows are freely available, promoting wider adoption and use.