How negative sampling shapes the performance of transcription factor binding site prediction models

Area of Science:

• Genomics
• Bioinformatics
• Computational Biology

Background:

• Transcription factors (TFs) regulate gene expression through DNA binding.
• Predicting TF binding sites (TFBSs) is vital for understanding gene regulation and development.
• Deep learning models for TFBS prediction often use ChIP-seq data, typically treated as positive samples.

Purpose of the Study:

• To investigate the impact of various negative sampling techniques on TFBS prediction performance.
• To evaluate common negative sampling strategies, including genomic, shuffling, and neighborhood sampling.
• To highlight the critical role of negative data selection in TFBS prediction model accuracy.

Main Methods:

• Created high-quality test datasets using ChIP-seq and ATAC-seq data.
• Trained prediction models using genomic sampling, shuffling, dinucleotide shuffling, neighborhood sampling, and cell line specific sampling.
• Simulated scenarios lacking matched ATAC-seq data to assess method robustness.

Main Results:

• Performance metrics on training datasets often inflate true model performance.
• Genomic sampling of negatives, based on similarity to positives, yielded the best results among tested techniques.
• Dinucleotide shuffled negatives, a common practice, resulted in poor model performance.

Conclusions:

• The selection of negative sampling techniques significantly impacts TFBS prediction model performance.
• Careful consideration of negative data is essential for reliable TFBS prediction.
• Standard practices like dinucleotide shuffling may not be optimal for TFBS prediction.