A simulation study of the reliability of recombination detection methods

Area of Science:

• Population Genetics
• Bioinformatics
• Molecular Evolution

Background:

• Numerous methods exist for detecting recombination and mosaic structures in DNA sequences.
• The reliability and performance of these detection methods are not fully understood.
• Understanding recombination detection is crucial for accurate phylogenetic inference and evolutionary studies.

Purpose of the Study:

• To assess the power of four different DNA sequence recombination detection methods.
• To investigate the impact of varying recombination and mutation rates on method performance.
• To examine how the shape of the underlying genealogy influences detection accuracy.

Main Methods:

• Simulated DNA sequence samples with controlled levels of recombination and mutation.
• Evaluation of four distinct recombination detection algorithms.
• Analysis of method performance across different genealogical tree shapes, including those from population expansion models.

Main Results:

• All investigated methods detected significantly fewer recombinations than theoretically possible.
• Methods explicitly utilizing the principle of incompatibility generally exhibited higher detection power.
• This enhanced power was particularly evident in phylogenies shaped by population expansion, characterized by long terminal branches and short internal branches.

Conclusions:

• Current DNA recombination detection methods have limitations in their sensitivity.
• Incompatibility-based methods offer a more robust approach to detecting recombination.
• Genealogy shape, influenced by demographic history, critically affects the performance of recombination detection tools.