Discriminative graph regularized representation learning for recognition

Feature extraction has been extensively studied in the machine learning field as it plays a critical role in the success of various practical applications. To uncover compact low-dimensional feature representations with strong generalization and discrimination capabilities for recognition tasks, in this paper, we present a novel discriminative graph regularized representation learning (DGRL) model that is able to elegantly incorporate both global and local geometric structures as well as the label structure of data into a joint framework. Specifically, DGRL first integrates dimension reduction into ridge regression rather than treated them as two irrelevant steps, which enables us to capture the underlying subspace structure and correlation patterns among classes. Additionally, a graph regularizer that fully utilizes the local class information is developed and introduced to the new framework so as to enhance the classification accuracy and prevent overfitting. A kernel version of DGRL, called KDGRL, is also established for dealing with complex nonlinear data by using the kernel trick. The proposed framework naturally unifies several well-known approaches and elucidates their intrinsic relationships. We provide detailed theoretical derivations of the resulting optimization problems of DGRL and KDGRL. Meanwhile, we design two simple and tractable parameter estimation procedures based on cross-validation technique to speed up the model selection processes for DGRL and KDGRL. Finally, we conduct comprehensive experiments on diverse benchmark databases drawn from different areas to evaluate the proposed theories and algorithms. The results well demonstrate the effectiveness and superiority of our methods.