A particle flow filter for high-dimensional system applications

A novel particle filter proposed recently, the particle flow filter (PFF), avoids the long-existing weight degeneracy problem in particle filters and, therefore, has great potential to be applied in high-dimensional systems. The PFF adopts the idea of a particle flow, which sequentially pushes the particles from the prior to the posterior distribution, without changing the weight of each particle. The essence of the PFF is that it assumes the particle flow is embedded in a reproducing kernel Hilbert space, so that a practical solution for the particle flow is obtained. The particle flow is independent of the choice of kernel in the limit of an infinite number of particles. Given a finite number of particles, we have found that a scalar kernel fails in high-dimensional and sparsely observed settings. A new matrix-valued kernel is proposed that prevents the collapse of the marginal distribution of observed variables in a high-dimensional system. The performance of the PFF is tested and compared with a well-tuned local ensemble transform Kalman filter (LETKF) using the 1,000-dimensional Lorenz 96 model. It is shown that the PFF is comparable to the LETKF for linear observations, except that explicit covariance inflation is not necessary for the PFF. For nonlinear observations, the PFF outperforms LETKF and is able to capture the multimodal likelihood behavior, demonstrating that the PFF is a viable path to fully nonlinear geophysical data assimilation.