PyBDA: a command line tool for automated analysis of big biological data sets

Area of Science:

• Computational Biology
• Bioinformatics
• Machine Learning

Background:

• Analyzing large, high-dimensional biological datasets presents computational challenges.
• Existing tools lack scalability for datasets with hundreds of millions of data points.

Purpose of the Study:

• To develop a scalable, automated tool for analyzing big biological data.
• To provide an accessible solution for bioinformaticians facing computational difficulties.

Main Methods:

• Developed PyBDA, a novel machine learning command-line tool.
• Utilized Apache Spark for distributed backend processing.
• Integrated Snakemake for automated job scheduling on high-performance computing clusters.

Main Results:

• PyBDA enables automated, distributed analysis of big biological datasets.
• The tool scales beyond the capacity of current applications.
• Successfully analyzed image-based RNA interference data from 150 million single cells.

Conclusions:

• PyBDA offers automated, user-friendly data analysis with common statistical methods and machine learning algorithms.
• Accessible via simple command-line calls for a broad user base.
• PyBDA is available at https://pybda.rtfd.io.