Prey group defense and hunting cooperation among generalist-predators induce complex dynamics: a mathematical study

Group defense in prey and hunting cooperation in predators are two important ecological phenomena and can occur concurrently. In this article, we consider cooperative hunting in generalist predators and group defense in prey under a mathematical framework to comprehend the enormous diversity the model could capture. To do so, we consider a modified Holling-Tanner model where we implement Holling type IV functional response to characterize grazing pattern of predators where prey species exhibit group defense. Additionally, we allow a modification in the attack rate of predators to quantify the hunting cooperation among them. The model admits three boundary equilibria and up to three coexistence equilibrium points. The geometry of the nontrivial prey and predator nullclines and thus the number of coexistence equilibria primarily depends on a specific threshold of the availability of alternative food for predators. We use linear stability analysis to determine the types of hyperbolic equilibrium points and characterize the non-hyperbolic equilibrium points through normal form and center manifold theory. Change in the model parameters leading to the occurrences of a series of local bifurcations from non-hyperbolic equilibrium points, namely, transcritical, saddle-node, Hopf, cusp and Bogdanov-Takens bifurcation; there are also occurrences of global bifurcations such as homoclinic bifurcation and saddle-node bifurcation of limit cycles. We observe two interesting closed 'bubble' form induced by global bifurcations due to change in the strength of hunting cooperation and the availability of alternative food for predators. A three dimensional bifurcation diagram, concerning the original system parameters, captures how the alternation in model formulation induces gradual changes in the bifurcation scenarios. Our model highlights the stabilizing effects of group or gregarious behaviour in both prey and predator, hence supporting the predator-herbivore regulation hypothesis. Additionally, our model highlights the occurrence of "saltatory equilibria" in ecological systems and capture the dynamics observed for lion-herbivore interactions.