Impact of Allee and fear effects in a fractional order prey–predator system with group defense and prey refuge

Abstract

This paper presents a novel fractional-order model of a prey–predator system that incorporates group defense and prey refuge mechanisms, along with Allee and fear effects. First, we examine the existence, uniqueness, non-negativity, and boundedness of the solution of the system. Second, a comprehensive analysis is conducted on the existence, stability, and coexistence of equilibrium states in the system, which are crucial for comprehending prey–predator system behavior. Our investigation reveals that the coexistence equilibrium undergoes a Hopf bifurcation under five key parameters. Specifically, an increased threshold for the transition between group and individual behavior, influenced by different strengths of the Allee effect, enhances the stability of both populations. This discovery sheds light on the role of group effects in shaping prey–predator interactions and ecosystem stability. Third, system discretization is employed to explore the impact of step size on stimulating stability and to investigate the Neimark–Sacker bifurcation, providing a more comprehensive understanding of system behavior. The role of step size as a constraint on stability is examined, revealing the system’s progression from stability to chaos. Consequently, our results offer a more flexible mechanism for adjusting the stability and dynamics of the two species. Finally, numerical simulations are utilized to validate the reasonableness of the research findings.