Diminishing spectral bias in physics-informed neural networks using spatially-adaptive Fourier feature encoding

Physics-informed neural networks (PINNs) have recently emerged as a promising framework for solving partial differential equation (PDE) systems in computer mechanics. However, PINNs still struggle in simulating systems whose solution functions exhibit high-frequency patterns, especially in cases with wide frequency spectrums. Current methods apply Fourier feature mappings to the input to improve the learning ability of model on high-frequency components. However, they are largely problem-dependent which require proper selection of hyperparameters and introduces additional training difficulty into the optimization. To this end, we present a spatially adaptive Fourier feature encoding method accompanied by a tree-based sampling strategy in this work. Specifically, we propose to guide the Fourier feature mappings of input by gradually exposing the input coordinate from low to higher encoding frequencies during training through the feedback loop of loss. Meanwhile, we also propose to refine the sampling of residual points by presenting a novel tree-based sampling strategy. This method represents the input domain by a tree and formulates the sampling of residual points as a resource allocation problem which optimizes the sampling of residual points during training and assigns more computational capacity to the underfit region. The effectiveness of our proposed method is demonstrated in several challenging PDE problems, including Poisson equation, heat equation, Navier-Stokes equations, Reynolds-Averaged Navier-Stokes equations, and Maxwell equation. The results indicate that our method can better allocate the computational resources during training and enable the model to fit the local frequencies of target function adaptively.