Extending the IICR to complex nonstationary structured models

Population genetic studies use genetic data to understand aspects of the evolutionary history of species. Here, we extend the piecewise-homogeneous structured coalescent framework to complex models where the parameters of the model, and thus the state space, change between periods of constant gene flow. To do this, we introduce glue matrices to map one state space to another, hence enabling the computation of the inverse instantaneous coalescence rate (IICR) using Q-matrices of different dimensions. This approach allows us to study structured models where the number of demes changes forward in time due to extinction or foundation of demes. Our analysis confirms that interpreting IICR curves as indicators of changes in Ne can be misleading and that a robust theoretical framework is required to ensure the correct interpretation of coalescent-based inferential methods. We show that there are cases where new deme foundations can generate an increase in the IICR (forward in time) hence producing an intuitive change where an increase in the IICR follows an increase in the size of the population. However, when we explore the impact of changes in migration rates, timing of deme creation or deletion, and sampling strategies on the IICR, we also identify several counter-intuitive results , where changes in the IICR and the total size of the population appear disconnected. For instance, we find that the IICR starts to decrease several generations before any change in the number of demes, effectively anticipating the upcoming change. By presenting examples of IICRs for various transitions and sampling scenarios, we emphasize the importance of a thorough understanding of the IICR as a foundation for accurate demographic inference.