Related Experiment Video
Updated: Jul 25, 2026
10:41
Leveraging CyVerse Resources for De Novo Comparative Transcriptomics of Underserved (Non-model) Organisms
Published on: May 9, 2017
Celeste: A cloud-based genomics infrastructure with variant-calling pipeline suited for population-scale sequencing
Summary
Celeste, an open-source cloud architecture, successfully analyzes large-scale genomic data for the All of Us Research Program. This scalable infrastructure enhances whole-genome sequencing analysis, enabling diverse research initiatives.
Area of Science:
- Genomics
- Bioinformatics
- Cloud Computing
Background:
- The All of Us Research Program is a major initiative generating genetic data from over one million diverse individuals.
- This program integrates extensive genetic data with longitudinal health information to create novel research platforms.
- Celeste is an open-source cloud architecture designed for large-scale genomics workflows.
Purpose of the Study:
- To describe the design and implementation of Celeste, a cloud architecture for genomics.
- To demonstrate Celeste's capability in analyzing petabytes of genomic data for the All of Us Research Program.
- To provide a scalable and resilient infrastructure for large-scale sequencing efforts.
Main Methods:
- Utilizing serverless technology and container orchestration for data management.
- Implementing a scalable cloud architecture on Amazon Web Services (AWS).
- Adapting and tuning variant-calling pipelines for clinical significance and accuracy.
Main Results:
- Celeste analyzed petabytes of genomic data, processing up to 9,000 whole-genome sequencing samples monthly.
- In 12 months, the system coordinated over 1.8 million bioinformatics, quality control, and clinical reporting jobs.
- Optimized variant-calling pipelines improved precision and accuracy for clinically significant variants.
Conclusions:
- Celeste offers significant improvements in turn-around time, scalability, security, and storage for whole-genome sequencing analysis.
- The Celeste framework provides a harmonized pipeline standard for population-scale sequencing to meet clinical requirements.
- Celeste is available as an AWS deployment on GitHub, including command-line parameters and software containers.
Related Concept Videos
02:02Genomics
Genomics is the science of genomes: it is the study of all the genetic material of an organism. In humans, the genome consists of information carried in 23 pairs of chromosomes in the nucleus, as well as mitochondrial DNA. In genomics, both coding and non-coding DNA is sequenced and analyzed. Genomics allows a better understanding of all living things, their evolution, and their diversity. It has a myriad of uses: for example, to build phylogenetic trees, to improve productivity and...
01:57Sanger Sequencing
DNA sequencing is a fundamental technique that is routinely used in the biological sciences. This method can be applied to a range of questions at different scales - from the sequencing of a cloned DNA fragment or the study of a mutation in a gene up to whole-genome sequencing. However, despite the widespread use of sequencing today, it was not until 1977 that Fredrick Sanger and his collaborators developed the chain-termination method to decode DNA sequences. It relies on the separation of a...
03:00Next-generation Sequencing
The first human genome sequencing project cost $2.7 billion and was declared complete in 2003, after 15 years of international cooperation and collaboration between several research teams and funding agencies. Today, with the advent of next-generation sequencing technologies, the cost and time of sequencing a human genome have dropped over 100 fold.
Next-Generation Sequencing Methods
Although all next-generation methods use different technologies, they all share a set of standard features.
Next-Generation Sequencing Methods
Although all next-generation methods use different technologies, they all share a set of standard features.
03:21RNA-seq
RNA sequencing, or RNA-Seq, is a high-throughput sequencing technology used to study the transcriptome of a cell. Transcriptomics helps to interpret the functional elements of a genome and identify the molecular constituents of an organism. Additionally, it also helps in understanding the development of an organism and the occurrence of diseases.
Before the discovery of RNA-seq, microarray-based methods and Sanger sequencing were used for transcriptome analysis. However, while microarray-based...
Before the discovery of RNA-seq, microarray-based methods and Sanger sequencing were used for transcriptome analysis. However, while microarray-based...