Evolving from bioinformatics in-the-small to bioinformatics in-the-large

Area of Science:

• Bioinformatics
• Computational Biology
• Software Engineering

Background:

• Traditional bioinformatics often focuses on individual data and tools, leading to fragmented systems.
• Incompatibility issues arise from small-scale approaches, making integrated systems difficult to build.
• The complexity of integrating diverse bioinformatics tools necessitates a shift in perspective.

Purpose of the Study:

• To advocate for a fundamental shift in bioinformatics from an "in-the-small" to an "in-the-large" perspective.
• To demonstrate how large-scale engineering principles can address integration challenges in bioinformatics.
• To highlight the importance of metadata and dependency management in large-scale bioinformatics.

Main Methods:

• Analysis of common problems in small-scale bioinformatics tool integration.
• Examination of the "make" utility as a model for dependency management.
• Case studies of two large-scale bioinformatics projects illustrating the "in-the-large" approach.

Main Results:

• The "in-the-large" perspective effectively manages complexity and overcomes tool incompatibility.
• Explicitly defining and managing knowledge flows and knowledge maps is crucial.
• Large-scale engineering principles from software engineering and data management are applicable to bioinformatics.

Conclusions:

• A paradigm shift to "bioinformatics in-the-large" is essential for future development.
• Managing metadata and dependencies is key to successful large-scale bioinformatics integration.
• Adopting an "in-the-large" perspective provides a significant advantage in complex bioinformatics endeavors.