Big Omics Data Experience
Patricia Kovatch1, Anthony Costa1, Zachary Giles1
1Icahn School of Medicine at Mount Sinai, 1 Gustave L. Levy Place, New York, NY 10029, 212-241-6500.
SC ... Conference Proceedings. SC (Conference : Supercomputing)
|February 22, 2019
Summary
Optimizing genomic data analysis requires tailored systems. This study details a high-throughput system design for the Genome Analysis ToolKit (GATK) pipeline, enhancing compute and storage efficiency for personalized medicine workflows.
Area of Science:
- Genomics
- Bioinformatics
- Computational Biology
Background:
- Personalized medicine drives increased human genome sequencing.
- This surge necessitates significant advancements in compute and storage infrastructure.
- Existing systems may not be optimized for the unique demands of genomic workloads.
Purpose of the Study:
- To design a high-throughput system for genomic data analysis without altering standard software.
- To optimize the Genome Analysis ToolKit (GATK) Best Practices pipeline for DNA and RNA sequencing.
- To improve the efficiency of compute and storage resources for genomic research.
Main Methods:
- Analysis of usage statistics, benchmarks, and available technologies.
- Evaluation of compute, workload, and I/O characteristics specific to genomic pipelines.
- System design focused on maximizing throughput for the GATK whole genome pipeline.
Main Results:
- Genomic workloads exhibit distinct characteristics compared to traditional High-Performance Computing (HPC).
- Specific configurations of schedulers and I/O subsystems are crucial for reliability and performance.
- The designed system achieved faster, repeatable performance and more efficient resource utilization.
Conclusions:
- Tailored system configurations are essential for optimizing genomic data analysis.
- Understanding user workflows (researchers and clinicians) is key to effective system design.
- The developed system enhances efficiency and performance for personalized medicine initiatives.
Related Concept Videos
01:13Data Collection by Experiments
27.3K
Data collection is a systematic method of obtaining, observing, measuring, and analyzing accurate information. An experimental study is a standard method of data collection that involves the manipulation of the samples by applying some form of treatment prior to data collection. It refers to manipulating one variable to determine its changes on another variable. The sample subjected to treatment is known as “experimental units.”
An example of the experimental method is a public...
An example of the experimental method is a public...
27.3K
01:12What is an Experiment?
17.8K
An experiment is a planned activity carried out under controlled conditions. The purpose of an experiment is to investigate the relationship between two variables. When one variable causes change in another, we call the first variable the explanatory or independent variable. The affected variable is called the response or dependent variable. In a randomized experiment, the researcher manipulates values of the explanatory variable and measures the resulting changes in the response variable. The...
17.8K
01:19Thomson's e/m Experiment
6.7K
In a beam of charged particles created by a heated cathode, the particles move at different speeds. However, many applications need a beam with uniform particle speeds. An arrangement known as a velocity selector uses electric and magnetic fields to pick particles with a particular speed from the beam.
A particle with charge q, speed v, and mass m enters an area from the top, where the magnetic and electric fields are perpendicular both to the particle's motion and to one another. The magnetic...
A particle with charge q, speed v, and mass m enters an area from the top, where the magnetic and electric fields are perpendicular both to the particle's motion and to one another. The magnetic...
6.7K
01:11How Data are Classified: Categorical Data
44.2K
A variable, usually notated by capital letters such as X and Y, is a characteristic or measurement that can be determined for each member of a population. Data are the actual values of variables. They may be numbers, or they may be words. Datum is a single value.
Data are classified based on whether they are measurable or not. Categorical data cannot be measured; instead, it can be divided into categories. For example, if Y denotes a person's party affiliation, some examples of Y include...
Data are classified based on whether they are measurable or not. Categorical data cannot be measured; instead, it can be divided into categories. For example, if Y denotes a person's party affiliation, some examples of Y include...
44.2K
00:59How Data are Classified: Numerical Data
37.7K
Data that are countable or measurable in specific units are called numerical or quantitative data. Quantitative data are always numbers. Quantitative data are the result of counting or measuring the attributes of a population. Amount of money, pulse rate, weight, number of people living in a town, and number of students who opt for statistics are examples of quantitative data.
Quantitative data may be either discrete or continuous. All quantitative data that take on only specific numerical...
Quantitative data may be either discrete or continuous. All quantitative data that take on only specific numerical...
37.7K
01:13Controls in Experiments
16.8K
When conducting an experiment, it is crucial to have control to reduce bias and accurately measure the dependent variables. It also marks the results more reliable. Controls are elements in an experiment that have the same characteristics as the treatment groups but are not affected by the independent variable. By sorting these data into control and experimental conditions, the relationship between the dependent and independent variables can be drawn. A randomized experiment always includes a...
16.8K