Cooler: scalable storage for Hi-C data and other genomically labeled arrays

Area of Science:

• Genomics
• Bioinformatics
• Computational Biology

Background:

• Existing epigenomic datasets are primarily one-dimensional.
• Newer technologies generate two-dimensional genomic interaction maps.
• Dense storage of these maps incurs high computational and storage costs.

Purpose of the Study:

• To develop a data storage strategy for multidimensional genomic datasets.
• To address the need for efficient compression and random access in genomic data.
• To facilitate the development of scalable algorithms for genomic data analysis.

Main Methods:

• Developed a sparse data model file format named cooler.
• Utilized HDF5 as the underlying storage technology.
• Created a Python library and command-line tools for cooler data manipulation.

Main Results:

• Cooler supports genomically labeled matrices at any resolution.
• The format accommodates diverse data descriptions, resolutions, and metadata.
• Cooler has been adopted as a standard by the NIH 4D Nucleome Consortium.

Conclusions:

• Cooler provides an efficient solution for storing and accessing 2D genomic interaction data.
• The format's flexibility and efficiency support scalable genomic data analysis.
• Cooler is a valuable tool for researchers working with large-scale epigenomic datasets.