SNPFile--a software library and file format for large scale association mapping and population genetics studies

Area of Science:

• Genomics
• Bioinformatics
• Computational Biology

Background:

• High-throughput genotyping generates massive datasets for genome-wide studies.
• Existing data storage solutions (spreadsheets, text files) are inadequate for large-scale genomic data.
• Efficient data management is crucial for handling millions of genetic markers and thousands of individuals.

Purpose of the Study:

• To introduce a novel binary file format for storing Single Nucleotide Polymorphism (SNP) data.
• To develop a software library for efficient manipulation of this new data format.
• To address the informatics challenges posed by large-scale genomic datasets.

Main Methods:

• Development of a new binary file format optimized for SNP data.
• Implementation of a software library for file I/O and data manipulation.
• Design of a flexible serialization mechanism for incorporating additional data types.

Main Results:

• The new binary format efficiently stores genotype data and associated information.
• The format is optimized for Input/Output (IO) efficiency, particularly for multi-locus analyses.
• Significant reduction in informatics burden and simplification of analysis runs were observed in internal studies.

Conclusions:

• The developed binary file format offers substantial benefits for managing and analyzing large genomic datasets.
• A key limitation is the current limited support by external analysis tools.
• Scripting interfaces are available to facilitate data conversion, mitigating compatibility issues.