Interpretable inverse iteration mean shift networks for clustering tasks

Neural networks have become the standard approach for tasks such as computer vision, machine translation and pattern recognition. While they exhibit significant feature representation capabilities, they often lack interpretability. This suggests that it might be beneficial to explore interpretable machine learning approaches in order to inform the neural network architectures. On the contrary, the mean shift algorithm (MS) has an unambiguous computational process, yet lacking in representation power. In order to draw on the advantages of both neural networks and the mean shift method, we propose the mean shift network (MS-Net) - a novel architecture, which is an inverse iteration fuzzy clustering network. Each layer of the proposed network possesses good interpretability, while the network as a whole is able to produce strong feature representations. To relax the limitations of the kernel function and ensure convergence, we design a continuously-differentiable Gaussian-inspired kernel as the activation function for the membership layer of MS-Net. Furthermore, we devise a weighted version of the architecture, called WMS-Net, to incorporate the importance of the training examples. We present theoretical results, with proofs of weak and strong convergence. We also consider two extensions to the proposed method, CB-MS-Net and CB-WMS-Net, which apply the curvature-based method to MS-Net and WMS-Net. Simulation results on 5 clustering tasks and 6 real-world datasets confirm the effectiveness of the proposed algorithms.