Pangenome-Informed Language Models for Synthetic Genome Sequence Generation

Area of Science:

• Genomics
• Bioinformatics
• Computational Biology

Background:

• Public genome datasets (e.g., Human Genome Project) have accelerated research.
• Protecting individual privacy in genomic data sharing is crucial.
• Current methods for synthetic data generation need improvement for complex genomic structures.

Purpose of the Study:

• To generate privacy-preserving synthetic DNA data using pangenomes and pretrained language models (PTLMs).
• To develop and evaluate novel tokenization schemes for enhanced DNA modeling.
• To ensure generated data complies with differential privacy (DP) standards.

Main Methods:

• Utilized pangenome graphs and PTLMs for synthetic DNA data generation.
• Introduced two novel pangenome-based tokenization schemes.
• Compared novel tokenization with single nucleotide and k-mer methods.
• Applied differential privacy (DP) techniques to the pangenome graph and training process.

Main Results:

• Novel tokenization schemes improved model performance consistency and effective context length compared to classical methods.
• The proposed DP method allows for privacy-compliant utilization of pangenome graphs.
• Evaluated the trade-off between privacy and accuracy in DP-trained models.

Conclusions:

• Pangenome-based tokenization with PTLMs offers a promising approach for privacy-preserving synthetic DNA data generation.
• The developed methods enhance the modeling of DNA sequences while adhering to DP standards.
• This work facilitates secure sharing of genomic information for downstream analysis.