Neural networks for self-adjusting mutation rate estimation when the recombination rate is unknown

Area of Science:

• Population Genetics
• Computational Biology
• Machine Learning

Background:

• Estimating mutation rates and effective population size is crucial in population genetics.
• Existing optimal linear estimation methods are well-established for low and high recombination rates.
• Intermediate recombination rates present challenges for accurate model-based estimation.

Purpose of the Study:

• To investigate the efficacy of feedforward neural networks for mutation rate estimation.
• To compare the performance of neural network estimators against established model-based estimators.
• To develop a robust estimation method applicable to varying or unknown local recombination rates.

Main Methods:

• Utilized feedforward neural networks trained on the site frequency spectrum.
• Compared neural network performance with model-based estimators (Fu, Futschik et al., Watterson).
• Employed model-based estimators to adjust training data weights for neural network optimization.

Main Results:

• Neural networks successfully reproduced existing model-based estimators.
• A single neural network estimator with adjusted weights performed comparably to model-based estimators across low and high recombination rates.
• The neural network approach demonstrated superior performance for intermediate recombination rates.

Conclusions:

• Feedforward neural networks offer a viable and effective tool for mutation rate estimation in population genetics.
• This machine learning approach provides a robust solution, particularly for scenarios with intermediate recombination rates.
• The method shows promise for real-world applications, including analysis of genomic data with varying recombination landscapes.