Interpretable facial landmark detection by multi-expert collaborative uncertainty-aware deep networks

In recent years, heatmap regression-based methods have become a dominant approach in facial landmark detection (FLD), demonstrating impressive performance. However, these methods generally predict landmark coordinates by optimizing heatmaps based on a predefined Gaussian distribution, which disregards the estimation of landmark uncertainty and also constrain both detection accuracy and interpretability. Furthermore, these methods still face challenges with faces under large poses and heavy occlusions, as they struggle to model effective facial shape constraints. To overcome these challenges, we propose a Multi-Expert Collaborative Uncertainty-Aware Deep Network (MCUDN) to achieve more robust and interpretable FLD. Specifically, we propose an Uncertainty-Aware Regression (UAR) method that adaptively adjusts the contribution of different landmarks based on their uncertainty during regression. By penalizing landmarks with higher uncertainty, the UAR method dynamically controls the gradient of localization during training, resulting in more accurate landmark detection. Moreover, a novel Multi-Expert Collaborative Learning (MECL) model is developed to extract multi-dependency collaborative features, enhancing facial shape constraints through multi-expert collaboration. Experimental results on challenging benchmark datasets demonstrate that integrating the UAR method and MECL model within the MCUDN framework yields a synergistic effect, outperforming current state-of-the-art methods.