Likelihood-based inferences under isolation by distance: two-dimensional habitats and confidence intervals

Area of Science:

• Population Genetics
• Computational Biology
• Ecological Modeling

Background:

• Likelihood-based inference methods for structured populations are established but under-tested in large networks.
• Previous software focused on linear habitats, limiting broader application.

Purpose of the Study:

• Extend inference software to two-dimensional habitats.
• Analyze confidence interval coverage properties in linear and 2D habitats.
• Evaluate the impact of model misspecification and approximations on inference.

Main Methods:

• Extended a previous software implementation for linear habitats to two-dimensional habitats.
• Analyzed standard likelihood and an efficient approximation method.
• Investigated effects of mutation model misspecification, dispersal distribution misspecification, and spatial binning of samples.

Main Results:

• Estimators showed low bias and mean square error with no model misspecification.
• Inferences for dispersal and mutation rate were sensitive to misspecification and coalescent approximations.
• Neighborhood parameter inference was robust to misspecification and spatial binning.
• Likelihood inference proved feasible and more efficient than moment-based methods in realistic conditions for up to 400 populations.

Conclusions:

• Likelihood-based inference is a viable and efficient tool for analyzing genetic data in moderately sized population networks.
• Neighborhood parameter estimation is reliable even with complicating factors like model misspecification.
• Coalescent approximations should be used cautiously for inferring dispersal distribution shape.