Multilayer decomposition and synchronization dynamics of nested hypergraphs

Synchronization is a core issue in the study of complex systems. Hypergraphs-based synchronization has received much attention due to its superiority of capturing higher-order relationship between different units in complex systems. In simple hypergraphs, using a weighted method, hyperedges can be transformed into corresponding maximal cliques of appropriate size. This process simplifies the synchronization problem of higher-order complex networks, allowing synchronization on weighted networks. However, this approach is not suitable for nested hypergraphs. In this work, we focus on the case of diffusive coupling and propose a novel framework based on the master stability method to address this limitation. Specifically, we transform the nested hypergraph into a multilayer simple hypergraph, and further map it onto a multilayer weighted graph, allowing for an equivalent description of higher-order dynamics near the synchronization manifold. The core of this multilayer transformation method lies in its ability to preserve the interaction relationships within the nested hypergraph while simplifying its higher-order structure using layer-by-layer decomposition approach. Furthermore, through numerical experiments, this paper reveals the synchronization behaviors of nested hypergraphs under different coupling strengths, particularly phenomena such as synchronization reversals, cluster synchronization, complete synchronization, and amplitude death. The proposed method provides a new perspective and theoretical framework for understanding and analyzing synchronization dynamics in multilayer, nested structural networks.